the error occurred while selecting feature using recursive feature elimination in sklearn

Question

I tried to rank the feature using recursive feature elimination in sklearn. However, I got this error when using RFE. here are the error and code information.

  from sklearn import svm
  x_vals = data['all_data'][:,0:320]

 y_vals_new = np.array([0 if each=='Neg'  else 1 if each =='Neu' else 2 for each in data['all_data'][:,320]])

 clf = svm.SVC(decision_function_shape='ovo',kernel='rbf') 


rfe = RFE(clf, 200)
rfe = rfe.fit(x_vals,y_vals_new)

print(rfe.support_)
print(rfe.ranking_)

clf.fit(DEAP_x_train, DEAP_y_train) 

print("###Valence###")
print("when the kernel function is rbf")
print('The mean square error %10.9f ' % np.mean((clf.predict(DEAP_x_test)-DEAP_y_test)**2))  # The mean square error

print('the mean accuracy on the given test data and labels %10.9f'% clf.score(DEAP_x_test, DEAP_y_test)) 

Did anyone know the reason of this error？ some opinion showed that RFE（recursive feature elimination) only works with SVC when the kernel is chosen to be linear. Is that correct？ Thanks a lot！

Answer 1

Recursive feature elimination works by considering co-efficient value of feature column (in case of linear models) or variable importance exposed, by say, random forest classifier.

For example, in case of a linear model, it will eliminate feature with smallest (absolute) value of co-efficient (i.e. less impact on outcome).

In case of SVMs with non-linear kernels, the classification is happening in a transformed space of features and not original features. So it is not possible to assign importance measure to original features. So RFE doesn't work with RBF kernel here.

Related question on variable weight for non-linear kernel