2
$\begingroup$

I want to get mutual information in iris dataset to select best feature but i confused about mutual information.

  • What is concept of mutual information for selecting feature? Can anyone explain it in simple way?

You do not really understand something unless you can explain it to your grandmother.

Albert Einstein

$\endgroup$
  • $\begingroup$ Though I admire great explainer, I'm just pointing out that it is highly unlikely that Einstein said that. $\endgroup$ – Siong Thye Goh Nov 29 '18 at 17:00
1
$\begingroup$

In simple terms, mutual information is used to measure relevance and redundancy of features with primary objective of high accuracy, with least possible time complexity (computational loss). Generally either a filter or a wrapper approach is taken for feature extraction. A good understanding of Cross-Entropy and Information Theory shall be helpful to understand MI concept. This paper has a proposed feature subset selection algorithm, based on (conditional) mutual information (along with LNC estimator) computation that you may quickly run through.

$\endgroup$
1
$\begingroup$

Mutual information (MI) quantifies the amount of information needed to express one variable with the help of another variable. If two variables are independent of each other, their MI is zero, else it will be greater than zero. MRMR, a mutual information based feature selection uses MI, which considers a feature effective if it has maximum MI with its class label(maximum relevance) and minimum MI with rest of the features(minimum redundancy). This combination of maximum relevance and minimum redundancy will ensure better performance with smaller feature dimension.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.