0
$\begingroup$

I am trying to make predictions (using Weka) on a tabular dataset. It is a categorical dataset which is encoded by label encoder.

I got a good result for SVM and Logistic Regression, namely the accuracy is around 85%.

The dataset is high-dimensional and I like to fine-tune my accuracy.

So, I am thinking about the feature selection method. I found different feature selection techniques, such as CfsSubsetEval, Classifier Attribute eval, classifier subset eval, Cv attribute eval, Gain ratio attribute eval, Info gain attribute eval, OneRattribute eval, principal component, relief f attribute eval, Symmetric uncertainty, Wrapper subset eval.

I would like to know which one would be the best for the dataset that shows good accuracy with Logistic Regression or SVM?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

I don't think that there is a single feature selection method that works best with a specific algorithm, what they do is selecting the best features based on various criteria. These features can be useful or not to the algorithm that does the classification, regardless what this algorithm is.

Without knowing anything about your data or their distribution, you can simply try a lot of those methods to see which produces the best results, and see if these generalize with the test set.

Also, SVM itself can be used for feature selection, since it finds the optimal coefficient for each feature. I don't know if you can access those coefficients through Weka (sorry, not familiar with the software), but if you could they can be an indicator of how important each feature is.

$\endgroup$
3
  • $\begingroup$ Thank you for the answer? What do you mean by data distribution? How can I check it? I am also familiar with Python,So you can give me a hint in pandas as well. $\endgroup$
    – Encipher
    Oct 15, 2022 at 8:51
  • $\begingroup$ For machine learning in python most people use scikit-learn , it has implementations of both algorithms you mentioned. In SVM there is the attribute coef_ but now that I am thinking of it it might work only for linear kernel. $\endgroup$
    – liakoyras
    Oct 15, 2022 at 9:01
  • $\begingroup$ You can think of distribution as a function that contains all of the possible values of y. There are different ideal distributions (your data could approximate one of them), for example in normal distribution most values would be closer to the center (a bell curve). The distribution of your data can help you find some insights that might help you chose better algorithms or parameters. $\endgroup$
    – liakoyras
    Oct 15, 2022 at 9:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.