I have a dataset with more than 30 features and 50K of samples; I want to know how increasing/decreasing the number of features increases the ML performance. I use SelectKBest, RFE, and SelectFromModel to extract features.

When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the max_features value (10, 15, 20, 25 in my case)

My question is: does XGBClassifier though that there are only five useful features in my dataset? Or did I mistakenly implement the algorithm? If yes, what is the best practice?

from sklearn.feature_selection  import SelectFromModel
from xgboost                    import XGBClassifier

#Step 1
#Normalizing and preprocessing data

#Step 2
#I repeat the experiment when max_features equals to 10, 15, 20, 25
sf=SelectFromModel(XGBClassifier(), max_features=10)


#Step3
#Applying ML algorithms for selected features

Another point worth mentioning is when I apply the ML algorithm (Just for the five selected features), The AUC ROC is usually above 90% (excellent result).

When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the max_features value

My question is: does XGBClassifier though that there are only five useful features in my dataset?

from sklearn.feature_selection  import SelectFromModel
from xgboost                    import XGBClassifier

sf=SelectFromModel(XGBClassifier(), max_features=10).fit(X, y)


#The output only contains five True, all remaining are False
print(sf.get_support())
```

I have a dataset with more than 30 features and 50K of samples; I want to know how increasing/decreasing the number of features increases the ML performance. I use SelectKBest, RFE, and SelectFromModel to extract features.

When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the max_features value (10, 15, 20, 25 in my case)

My question is: does XGBClassifier though that there are only five useful features in my dataset? Or did I mistakenly implement the algorithm? If yes, what is the best practice?

from sklearn.feature_selection  import SelectFromModel
from xgboost                    import XGBClassifier

#Step 1
#Normalizing and preprocessing data

#Step 2
#I repeat the experiment when max_features equals to 10, 15, 20, 25
sf=SelectFromModel(XGBClassifier(), max_features=10)


#Step3
#Applying ML algorithms for selected features
```

I have a dataset with more than 30 features and 50K of samples; I want to know how increasing/decreasing the number of features increases the ML performance. I use SelectKBest, RFE, and SelectFromModel to extract features.

When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the max_features value (10, 15, 20, 25 in my case)

My question is: does XGBClassifier though that there are only five useful features in my dataset? Or did I mistakenly implement the algorithm? If yes, what is the best practice?

from sklearn.feature_selection  import SelectFromModel
from xgboost                    import XGBClassifier

#Step 1
#Normalizing and preprocessing data

#Step 2
#I repeat the experiment when max_features equals to 10, 15, 20, 25
sf=SelectFromModel(XGBClassifier(), max_features=10)


#Step3
#Applying ML algorithms for selected features

Another point worth mentioning is when I apply the ML algorithm (Just for the five selected features), The AUC ROC is usually above 90% (excellent result).