How to retrieve column names from applying a wrapper method in feature selection?

Question

This question probably has a simple answer to it, so I will get to the point...

How do I retrieve the names of the columns from applying a wrapper method in feature selection?

Code I have used:

from mlxtend.feature_selection import SequentialFeatureSelector as SFS

X = df[['A','B','C','D']].values
y = df[['F']].values

classifier = KNeighborsClassifier(n_neighbors=7) 
code = SFS(classifier,
           k_features=5, 
           forward=True, 
           floating=False, 
           verbose=8,
           scoring='accuracy'
)

code.fit(X, y)
code.k_feature_names_ 

Output:

Features: 3/3 -- score: 0.78

('1' '2', '3')

Therefore, how do I retrieve the names of the features ('1' '2', '3')?

Answer 1

Based on this scikit-learn documentation, you can get a boolean mask (in the same order) of the input features, via the get_support method:

Answer 2

By using .values in your definition of X, you've converted to a numpy array and lost the column names. Just removing that, you'll provide a frame to SRS, and mlxtend will use the column names in the k_feature_names_ attribute; so that's probably the best approach.

There are two other approaches: one is to add the custom_feature_names parameter of mlxtend.SequentialFeatureSelector to replicate the dataframe column names, and the other is to use the k_feature_idx_ parameter to slice a list of your columns. Both of these require you saving the column names somewhere before casting to numpy, so the first solution is generally easier.

See the first example, and the API documentation, on the mlxtend.SequentialFeatureSelector page.