0
$\begingroup$

When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the max_features value

My question is: does XGBClassifier though that there are only five useful features in my dataset?

from sklearn.feature_selection  import SelectFromModel
from xgboost                    import XGBClassifier

sf=SelectFromModel(XGBClassifier(), max_features=10).fit(X, y)


#The output only contains five True, all remaining are False
print(sf.get_support())
```
$\endgroup$
2
  • $\begingroup$ Can you provide a sample dataset that returns this issue? Then also, what is the output of sf.get_support() or sf.get_feature_names_out()? $\endgroup$
    – m13op22
    Nov 2, 2022 at 21:01
  • $\begingroup$ [False False False False False False False False False True False True False False False True False False False False True False False ] This is the output of sf.get_support() with four True. Relating the dataset, all features are numeric when I use SelectKBest or RFE it return the exact number (10) $\endgroup$
    – Niyaz
    Nov 2, 2022 at 21:16

1 Answer 1

1
$\begingroup$

To only select based on max_features, set threshold=-np.inf.

I found the above text in the documentation sklearn.feature_selection. This means as priority SelectFromModel depends on the threshold parameter and returns all features that pass the threshold (regardless of max_features).

If you want max_features fully function, then set threshold=-np.inf, in this case, all features pass the threshold, then max_features can select demanded features (based on their rank).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.