1
$\begingroup$

What does it mean if I used sklearn.feature_selection.SelectFromModel to select features and cross-validation score went up but the testset score went down. This was even more significant compared to when I did not use feature selection.

Does this mean it's causing more overfitting?

I thought feature selection is used to reduce overfitting.

$\endgroup$
1
  • $\begingroup$ Please share your code. $\endgroup$ Mar 27, 2021 at 13:41

1 Answer 1

1
$\begingroup$

Feature selection doesn't always mean reducing overfitting, feature selection is mainly used to reduce dimensionality. When we remove the least important features from the model, we reduce the model’s complexity and some noise in the data. Doing so will help to further mitigate overfitting but it's not always the case. Feature selection determines where the appropriate decision boundaries should be, and their complexity. It affects fitting and sampling, and classifier performance, but feature selection cannot always prevent overfitting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.