Why is ElasticNet performs worse than both Lasso and Ridge?

Question

I am using the following codes to build a few models on the same dataset:

X_train, X_test, y_train, y_test = train_test_split(X_in, y, test_size=0.25, random_state=42)

# Lasso regression
lasso = linear_model.Lasso()
lasso.fit(X_train, y_train)
pred_lasso = lasso.predict(X_test)

# Ridge regression
ridge = linear_model.Ridge()
ridge.fit(X_train, y_train)
pred_ridge = ridge.predict(X_test)

# ElasticNet
elastic = linear_model.ElasticNet()
elastic.fit(X_train, y_train)
pred_elastic = elastic.predict(X_test)

# R^2 Evaluation
print('R^2 for Lasso', r2_score(y_test, pred_lasso))
print('R^2 for Ridge', r2_score(y_test, pred_ridge))
print('R^2 for ElasticNet', r2_score(y_test, pred_elastic))
print('\n')

And the r2_score for the 3 models are:

R^2 for Lasso 0.28
R^2 for Ridge 0.14
R^2 for ElasticNet 0.02

This is confusing to me ... shouldn't the ElasticNet result fall somewhere between Lasso and Ridge? Why is ElasticNet result actually worse than the other two? Thanks!

Answer 1

The ElasticNet model is not being tuned. By default in scikit-learn, ElasticNet's l1_ratio parameter, the mixture of L1 and L2 penalty, is set to .5. A .5 l1_ratio represents an even mixture of L1 and L2 penalty and does not fit the data very well. Best practice is a cross-validation grid search for the optimal value of l1_ratio.