This repository has been archived by the owner on Dec 6, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 213
FistaRegressor does not converge for real data #151
Comments
Could you indicate how you call |
I update the code above to show how I called the |
Hello @arose13, I think it may have to do with a wrong choice of The following gives me identical results for lightning and sklearn: from scipy import stats
import matplotlib.pyplot as plt
import numpy as np
from lightning.regression import FistaRegressor
from sklearn.linear_model import LassoCV
from statsmodels.tools.tools import add_constant
from sklearn.datasets import load_boston
fig, axarr = plt.subplots(1, 2)
for ax, dataset in zip(axarr, ["simu", "boston"]):
if dataset == "simu":
N, P = 300, 30
m_true = np.zeros(P)
m_true[:4] = [2, -2, 2, 3]
noise = 3 * stats.norm().rvs(N)
data = 3*stats.norm().rvs((N, P))
target = data @ m_true + noise
else:
data, target = load_boston(return_X_y=True)
data = add_constant(data)
N = data.shape[0]
clf_sklearn = LassoCV(fit_intercept=False, tol=1e-10).fit(data, target)
fista = FistaRegressor(
C=1/N,
penalty='l1',
alpha=clf_sklearn.alpha_,
max_iter=1_000,
max_steps=1_000,
)
fista.fit(data, target)
ax.plot(clf_sklearn.coef_, label="sklearn")
ax.plot(fista.coef_[0], label="Lightning fista", linestyle='--')
ax.legend()
ax.set_title(dataset)
plt.show(block=False) |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
I can get the
FistaRegressor
to converge when data is trivial. The code for the simulated data is below.(X axis are the individual coefficients and the Y axis are the fitted coef magnitude)
But if I use real data I cannot get it to converge at all.
(X axis are the individual coefficients and the Y axis are the fitted coef magnitude)
I assume it has something to do with feature scaling but I'm not sure.
The text was updated successfully, but these errors were encountered: