0
$\begingroup$

I am using lasso for feature selection. I have selected lasso parameters from grid search. Now, I used the the best model which I got to fit on training data set. Thus, out of 70 variables I got 66. I observed if I fit lasso again on the same training data set, the variables get dropped again (by drop I mean their coefficients are zero). My doubt is should I go ahead with first iteration where I got 66 variables or should I fit again and again when there is no further variable drop?

$\endgroup$
1
$\begingroup$

In principle, that shouldn't happen. If you rerun Lasso on the same data set, with the same hyperparameters, after dropping attributes whose coefficient was set to zero in your first Lasso run, you should get the same result. If you see something different, something has gone wrong.

You'll need to investigate deeper to see if you can figure out why that is happening. Perhaps there is a bug in your code, or perhaps you are not providing the right settings to your implementation of Lasso, or perhaps the optimization algorithm is not finding the exact optimum solution, or perhaps something else has gone wrong. It's hard to say with the level of detail you have provided.

If you're using different hyperparameters the second time you run Lasso, anything could happen. Don't do that; that doesn't make sense.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.