Feature importance ratio

Question

I trained a Random Forest classifier (sklearn) and consequently computed the feature importance and consequently ranked them. The forest has 100 estimators. My top 5 features with their importances are as here:

f1 = 0.91
f2 = 0.04
f3 = 0.013
f4 = 0.007
f5 = 0.004

To avoid over-fitting, I did the evaluation using cross-validation and learning curve.

My question is that the importance for f1 seems significantly higher than other features. Does it imply incorrectness (over-fitting?) of any sort? Should I do feature selection in some other way to generalize the model better?

Answer 1

In fact, this means that the quality of the feature f1 is very high. Usually, you should be worried if you get all the features giving the same importance level, and that level is low.

It is very important to understand that random forest does two level of randomization: at the data level and the feature level and it is very difficult to over fit.

Since you have this large number of trees in the forest, I do not think you have over fitting problem.