I have some features and I am using Weka to classify my instances.

For example I have:

Number of adj number of adverb number of punctuation

in my feature set. However, I would like to know the contribution of each feature in the feature set. So what metrics or parameters are helpful to get the contribution of features?


This is called feature ranking, which is closely related to feature selection.

  • feature ranking = determining the importance of any individual feature
  • feature selection = selecting a subset of relevant features for use in model construction.

So if you are able to ranked features, you can use it to select features, and if you can select a subset of useful features, you've done at least a partial ranking by removing the useless ones.

This Wikipedia page and this Quora post should give some ideas. The distinction filter methods vs. wrapper based methods vs. embedded methods is the most common one.

One straightforward approximate way is to use feature importance with forests of trees:

enter image description here

Other common ways:

If you use scikit-learn, check out module-sklearn.feature_selection. I'd guess Weka has some similar functions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.