1
$\begingroup$

I have some feature sets say X1 and X2 ... Each feature set have some variable amount of features and there is no intersection between different feature sets Say X1 have 100 features and X2 have 500 features and none of the features they had is same . Though the file from which these features are extracted is same for both x1 and x2.

Each feature set can be used individually for binary classification. Like X1 can also be used for classification and X2 can also be used for classification. They can also be used together combined i.e X1 U X2 which will have 600 features

But I want to compare the feature sets and hence want to know if there is any statistical method which can be used to rank these feature sets other than classification accuracy

Like X1 is better than X2 and so on ...

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You could use some statistical method to rank these feature sets, such as extracting in each features set a discriminative score per feature (kolmogorov smirnof, mutual information, ...) and then take the mean, the median, the p95, ...

But, depending on the classifier that will be used, this score may be useless, because some classifiers act as features selectors (decision trees, neural networks, ...) and others don't (Gaussian NB, kNN,...).

Example
If you have 10 features in your 500 that are really good to discriminate your classification and the other 490 that do not contain information. Using a decision tree based classifier will lead to good performance as probably only the 10 features will be used. kNN in the other hand will lead to bad performance...
However, your features set score would remain the same.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hey, @etiennedm I also thought of that but found this method to be inefficient. I actually have 10 feature sets and wanted to use some combination of them, after which I will apply feature selection technique, so that's why wanted to rank them and then make combos. Do u have any method in mind which can help me achieve that? $\endgroup$
    – kartik khariwal
    Dec 15, 2020 at 16:40
  • $\begingroup$ Sorry, I am not aware of a technique that gives a multivariate score of a features set without either univariate scores (like I described in my answer) or a classifier approach (such as xgboost features selection). $\endgroup$
    – etiennedm
    Dec 16, 2020 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.