Select the best feature selection method for classification

Question

I am trying to make predictions (using Weka) on a tabular dataset. It is a categorical dataset which is encoded by label encoder.

I got a good result for SVM and Logistic Regression, namely the accuracy is around 85%.

The dataset is high-dimensional and I like to fine-tune my accuracy.

So, I am thinking about the feature selection method. I found different feature selection techniques, such as CfsSubsetEval, Classifier Attribute eval, classifier subset eval, Cv attribute eval, Gain ratio attribute eval, Info gain attribute eval, OneRattribute eval, principal component, relief f attribute eval, Symmetric uncertainty, Wrapper subset eval.

I would like to know which one would be the best for the dataset that shows good accuracy with Logistic Regression or SVM?

Answer 1

I don't think that there is a single feature selection method that works best with a specific algorithm, what they do is selecting the best features based on various criteria. These features can be useful or not to the algorithm that does the classification, regardless what this algorithm is.

Without knowing anything about your data or their distribution, you can simply try a lot of those methods to see which produces the best results, and see if these generalize with the test set.

Also, SVM itself can be used for feature selection, since it finds the optimal coefficient for each feature. I don't know if you can access those coefficients through Weka (sorry, not familiar with the software), but if you could they can be an indicator of how important each feature is.