In one paper on ML I read that chi square distribution is used to reduce the number of features. In that paper, features are words. That paper is related to Sentiment Analysis, so we have "positive", "negative" and "neutral" category.

  • How to calculate chi square distribution in that case?

  • In Python there is scipy.stats.chisquare which gives chi_square value and a p_value. What do we do then with these two pieces of information?

  • What to do for example with word "good" as a feature?

  • How to calculate chi square distribution, and what to do with that?

  • What does it mean to exclude some feature from the set of features, because in that paper it is mentioned that we take n of them with top chi square.

I really don't know how to do it. If there is any paper or book or link to learn that, please tell me.


1 Answer 1


There are different ways for feature selection. A very good read in machinelearningmastery, to recap:

  • Univariate Selection.
  • Recursive Feature Elimination.
  • Principal Component Analysis.
  • Feature Importance.

Chi-Squared test For Feature Selection goes under the Univariate Selection method for non-negative features. My favorite explanation of chi-Squared in one photo taken from this blogpost is: enter image description here

As you can see scikit-learn has an implementation for feature_selection using chi2 (perhaps according to scipy.stats.chisquare) as was shown very briefly in the above-mentioned blog post.

If you want a more thorough explanation and details how test ranks features based on statistics according to chi2 distribution and p-value etc., and also how to build your own chi2 class for feature selection in Python see this great post. Obviously one can read about the basics of chi2 distribution and test in wikipedia.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.