I am using the following codes to build a few models on the same dataset:

X_train, X_test, y_train, y_test = train_test_split(X_in, y, test_size=0.25, random_state=42)

# Lasso regression
lasso = linear_model.Lasso()
lasso.fit(X_train, y_train)
pred_lasso = lasso.predict(X_test)

# Ridge regression
ridge = linear_model.Ridge()
ridge.fit(X_train, y_train)
pred_ridge = ridge.predict(X_test)

# ElasticNet
elastic = linear_model.ElasticNet()
elastic.fit(X_train, y_train)
pred_elastic = elastic.predict(X_test)

# R^2 Evaluation
print('R^2 for Lasso', r2_score(y_test, pred_lasso))
print('R^2 for Ridge', r2_score(y_test, pred_ridge))
print('R^2 for ElasticNet', r2_score(y_test, pred_elastic))

And the r2_score for the 3 models are:

R^2 for Lasso 0.28
R^2 for Ridge 0.14
R^2 for ElasticNet 0.02

This is confusing to me ... shouldn't the ElasticNet result fall somewhere between Lasso and Ridge? Why is ElasticNet result actually worse than the other two? Thanks!

  • $\begingroup$ It's worth comparing the regularization coefficients between the three. $\endgroup$ – Emre Aug 7 '18 at 23:51

The ElasticNet model is not being tuned. By default in scikit-learn, ElasticNet's l1_ratio parameter, the mixture of L1 and L2 penalty, is set to .5. A .5 l1_ratio represents an even mixture of L1 and L2 penalty and does not fit the data very well. Best practice is a cross-validation grid search for the optimal value of l1_ratio.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.