Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
From the version 0.24, the scikit-learn has new method 'SequentialFeatureSelector', which adds (forward selection) or removes (backward selection) features to form a feature subset in a greedy fashion. It lets us to select features in the 'forward stepwise selection' or 'backward stepwise selection', described in the book 'Introduction to Statistical Learning (ISLR)'.
To use the SequentialFeatureSelector, you need to put 'int' or 'float' value to the parameter n_features_to_select. If you don't write anything, half of feature numbers are automatically put into the parameter. However, according to ISLR, you can know how many number of variables are appropriate only after you test all number of parameters and get the best model of each number of parameters. The plots shown below is from the ISLR, which shows that you can figure out how many number of features are appropriate only after testing all numbers of predictors. You cannot figure it out beforehand.
So I think an input "the lowest adjusted r_squared" for the parameter n_features_to_select. By doing so, you can choose the number of features that has the lowest adjusted r_squared, which cannot be known beforehand.
Why is scikit-learn made in such a way that int or float must be put into the parameter?
For "why", I think it's just a new transformer that maybe didn't get thoroughly thought-out. The default of "half the features" in particular seems very odd to me.
A middle ground, that I think is more useful, is to select features until there is no (or little) further improvement. That's being implemented in PR20145. If they would also expose the scores in an attribute, you could post-process with this PR by setting tol=-np.inf (so that all the features would eventually get added) and then selecting the best exposed score. I don't see an Issue suggesting storing scores (as in RFECV.cv_results_), but the ranking has been suggested in Issue19583.