Would adding Elastic Net as an additional Benchmark add any value when LASSO is already an included benchmark?

Question

I am currently engaged in a research project with a collaborator in which he is proposing a novel learning algorithm for optimal variable selection, and exploring its computational, statistical, and asymptotic properties; while I am proposing and running the several benchmark methods with which to compare its performance when it and the benchmarks I come up with are all run on the same set of 260k synthetic datasets which my collaborator has generated via Monte Carlo Simulation.

I have so far decided on 3 Benchmarks: BM1 - LASSSO Regression, BM2 - Backward Stepwise Regression, and BM3 - Forward Stepwise Regression. I have been considering also adding on Elastic Net as another 4th Benchmark, but something tells me it wouldn't be worth the extra coding and debugging hours. Would including it add any significant value over just including LASSO?

Answer 1

No, the reason is because Elastic Net as a cross between the L1 and L2 norms, would only ever select a subset of the variables that LASSO would select, or if the penalty is extremely close to having its lambda penalty set to 1, it might select the exact same set of variables as LASSO does. However, what it will never do is select one or more variables that LASSO does not select, because it has the same objective (loss) function it seeks to minimize, the same type of penalty, and is being fit on the same dataset.