1
$\begingroup$

On a set of 9 features I have applied Recursive Feature Elimination (RFE) algorithm using SVM estimator, following approach from (1). When requesting a subset of size 1 to be found, then RFE returned feature X.

However, when I trained SVM over each feature individually, I found another feature Y to have higher accuracy than SVM trained over X.

I thought that RFE finds features with the highest accuracy.

Is my understanding of RFE wrong?

(1): Gene selection for cancer classification using support vector machines

$\endgroup$
1
$\begingroup$

No, RFE cannot guarantee that it finds the feature subset with optimal score.

As with most greedy processes, the point of RFE is to reduce the computational cost (fitting a model for each of the $2^m$ feature subsets), at the cost of perhaps not finding the actual optimum (but hopefully "close enough").

See also https://stats.stackexchange.com/questions/232461/question-about-recursive-feature-elimination

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.