0
$\begingroup$

I tried to rank the feature using recursive feature elimination in sklearn. However, I got this error when using RFE. here are the error and code information. enter image description here

  from sklearn import svm
  x_vals = data['all_data'][:,0:320]

 y_vals_new = np.array([0 if each=='Neg'  else 1 if each =='Neu' else 2 for each in data['all_data'][:,320]])

 clf = svm.SVC(decision_function_shape='ovo',kernel='rbf') 


rfe = RFE(clf, 200)
rfe = rfe.fit(x_vals,y_vals_new)

print(rfe.support_)
print(rfe.ranking_)

clf.fit(DEAP_x_train, DEAP_y_train) 

print("###Valence###")
print("when the kernel function is rbf")
print('The mean square error %10.9f ' % np.mean((clf.predict(DEAP_x_test)-DEAP_y_test)**2))  # The mean square error

print('the mean accuracy on the given test data and labels %10.9f'% clf.score(DEAP_x_test, DEAP_y_test))

Did anyone know the reason of this error？ some opinion showed that RFE（recursive feature elimination) only works with SVC when the kernel is chosen to be linear. Is that correct？ Thanks a lot！

Improve this question
$\endgroup$
0
0
$\begingroup$

Recursive feature elimination works by considering co-efficient value of feature column (in case of linear models) or variable importance exposed, by say, random forest classifier.

For example, in case of a linear model, it will eliminate feature with smallest (absolute) value of co-efficient (i.e. less impact on outcome).

In case of SVMs with non-linear kernels, the classification is happening in a transformed space of features and not original features. So it is not possible to assign importance measure to original features. So RFE doesn't work with RBF kernel here.

Related question on variable weight for non-linear kernel

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for yr replying. I see. and, I just used the SelectKBest for selecting the best k（100 and 200)(320 features in total） features. However, the accuracy didn't change at all. Do you have any idea about this isssue? $\endgroup$ – Irving.ren Aug 1 '18 at 11:17
  • $\begingroup$ If you are satisfied with explanation, accept the answer :-) . On your second question in comment: accuracy on train set or test set? Assuming test set, it indicates that more features are not predictive. There are fewer features (covered in 100 best) which influence the outcome, remaining features don't have much impact. Whenever possible, prefer model with lower number of features at same accuracy. Check Akaike information criterion (AIC). $\endgroup$ – hssay Aug 1 '18 at 11:23
  • $\begingroup$ Thanks. What I mean is selecting the best feature before separating the train and test set. Accuracy is training by train set and testing by test set. However, the result didn't change at all. Any comments for this question? $\endgroup$ – Irving.ren Aug 1 '18 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.