1
$\begingroup$

I trained a Random Forest classifier (sklearn) and consequently computed the feature importance and consequently ranked them. The forest has 100 estimators. My top 5 features with their importances are as here:

f1 = 0.91
f2 = 0.04
f3 = 0.013
f4 = 0.007
f5 = 0.004

To avoid over-fitting, I did the evaluation using cross-validation and learning curve.

My question is that the importance for f1 seems significantly higher than other features. Does it imply incorrectness (over-fitting?) of any sort? Should I do feature selection in some other way to generalize the model better?

$\endgroup$

1 Answer 1

1
$\begingroup$

In fact, this means that the quality of the feature f1 is very high. Usually, you should be worried if you get all the features giving the same importance level, and that level is low.

It is very important to understand that random forest does two level of randomization: at the data level and the feature level and it is very difficult to over fit.

Since you have this large number of trees in the forest, I do not think you have over fitting problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.