# Why is ElasticNet performs worse than both Lasso and Ridge?

I am using the following codes to build a few models on the same dataset:

X_train, X_test, y_train, y_test = train_test_split(X_in, y, test_size=0.25, random_state=42)

# Lasso regression
lasso = linear_model.Lasso()
lasso.fit(X_train, y_train)
pred_lasso = lasso.predict(X_test)

# Ridge regression
ridge = linear_model.Ridge()
ridge.fit(X_train, y_train)
pred_ridge = ridge.predict(X_test)

# ElasticNet
elastic = linear_model.ElasticNet()
elastic.fit(X_train, y_train)
pred_elastic = elastic.predict(X_test)

# R^2 Evaluation
print('R^2 for Lasso', r2_score(y_test, pred_lasso))
print('R^2 for Ridge', r2_score(y_test, pred_ridge))
print('R^2 for ElasticNet', r2_score(y_test, pred_elastic))
print('\n')


And the r2_score for the 3 models are:

R^2 for Lasso 0.28
R^2 for Ridge 0.14
R^2 for ElasticNet 0.02


This is confusing to me ... shouldn't the ElasticNet result fall somewhere between Lasso and Ridge? Why is ElasticNet result actually worse than the other two? Thanks!

• It's worth comparing the regularization coefficients between the three. – Emre Aug 7 '18 at 23:51

The ElasticNet model is not being tuned. By default in scikit-learn, ElasticNet's l1_ratio parameter, the mixture of L1 and L2 penalty, is set to .5. A .5 l1_ratio represents an even mixture of L1 and L2 penalty and does not fit the data very well. Best practice is a cross-validation grid search for the optimal value of l1_ratio.