Is Recursive Feature Elimination finding best features subset?

Question

On a set of 9 features I have applied Recursive Feature Elimination (RFE) algorithm using SVM estimator, following approach from (1). When requesting a subset of size 1 to be found, then RFE returned feature X.

However, when I trained SVM over each feature individually, I found another feature Y to have higher accuracy than SVM trained over X.

I thought that RFE finds features with the highest accuracy.

Is my understanding of RFE wrong?

(1): Gene selection for cancer classification using support vector machines

Answer 1

No, RFE cannot guarantee that it finds the feature subset with optimal score.

As with most greedy processes, the point of RFE is to reduce the computational cost (fitting a model for each of the $2^m$ feature subsets), at the cost of perhaps not finding the actual optimum (but hopefully "close enough").

See also https://stats.stackexchange.com/questions/232461/question-about-recursive-feature-elimination