2
$\begingroup$

I am currently engaged in a research project with a collaborator in which he is proposing a novel learning algorithm for optimal variable selection, and exploring its computational, statistical, and asymptotic properties; while I am proposing and running the several benchmark methods with which to compare its performance when it and the benchmarks I come up with are all run on the same set of 260k synthetic datasets which my collaborator has generated via Monte Carlo Simulation.

I have so far decided on 3 Benchmarks: BM1 - LASSSO Regression, BM2 - Backward Stepwise Regression, and BM3 - Forward Stepwise Regression. I have been considering also adding on Elastic Net as another 4th Benchmark, but something tells me it wouldn't be worth the extra coding and debugging hours. Would including it add any significant value over just including LASSO?

$\endgroup$

1 Answer 1

2
$\begingroup$

No, the reason is because Elastic Net as a cross between the L1 and L2 norms, would only ever select a subset of the variables that LASSO would select, or if the penalty is extremely close to having its lambda penalty set to 1, it might select the exact same set of variables as LASSO does. However, what it will never do is select one or more variables that LASSO does not select, because it has the same objective (loss) function it seeks to minimize, the same type of penalty, and is being fit on the same dataset.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.