Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I want to get mutual information in iris dataset to select best feature but i confused about mutual information.
You do not really understand something unless you can explain it to your grandmother.
Albert Einstein
Mutual information (MI) quantifies the amount of information needed to express one variable with the help of another variable. If two variables are independent of each other, their MI is zero, else it will be greater than zero. MRMR, a mutual information based feature selection uses MI, which considers a feature effective if it has maximum MI with its class label(maximum relevance) and minimum MI with rest of the features(minimum redundancy). This combination of maximum relevance and minimum redundancy will ensure better performance with smaller feature dimension.
In simple terms, mutual information is used to measure relevance and redundancy of features with primary objective of high accuracy, with least possible time complexity (computational loss). Generally either a filter or a wrapper approach is taken for feature extraction. A good understanding of Cross-Entropy and Information Theory shall be helpful to understand MI concept. This paper has a proposed feature subset selection algorithm, based on (conditional) mutual information (along with LNC estimator) computation that you may quickly run through.
