0
$\begingroup$

Consider a experiment to predict the Google-Play apps rating using a Random-Forest classifier with scikit-learn in Python. Three attributes 'Free', 'Size' and 'Category' are utilized to predict the apps rating. 'Rating' (label) is not continuous value, instead, grouped into two classes 0 and 1. Where 0 is below 4 star rating and 1 is above 4 rating. Through Random-Forest, feature significance of all three predictive attributes and the F1 Evaluation of the model is also calculated.

Firstly, lets suppose model omits the 'Size' as most significant feature, so what is implied here, having larger size or lower size of an app contribute to the rating? What If there is no ascending or descending order in the attribute, for instance if the 'Category' is most significant, then what category contributed the most?

Secondly, the scikit-learn calculates the evaluation for each individual labels. As there are two labels, 0 and 1, so model yields two separate F1 scores for these labels and also the overall F1 scores of entire model. Now consider, for label 0, F1 is 30% and label 1 its 75%. Whereas overall F1 of entire model is 55%. In general, F1 and all other evaluation metrics must be around 90% for good prediction model. But suppose a scenario where above 70% F1 is considered good. Can I claim that these above mentioned attributes are good predictor of label 1 as its individual F1 is 75% but not for label 0, because it has only 30% F1 individually. If yes, then is it means predictive model cant find any relation between attributes and label 0, but finds a considerable relation with label 1? Or I have to consider the overall F1 which is 55%, and claim there exists very little correctional between 'Rating' and predictive attributes for all the labels. In conclusion, not a good prediction model at entirety for all the labels?

$\endgroup$
0
$\begingroup$

Firstly, lets suppose model omits the 'Size' as most significant feature, so what is implied here, having larger size or lower size of an app contribute to the rating? What If there is no ascending or descending order in the attribute, for instance, if the 'Category' is most significant, then what category contributed the most?

Decision Tree splits the space based on a feature value.
Good feature importance implies, when the model used that feature to split the space, split were cleaner [better Gini drop].
At this point, you can't know the value-specific correlation i.e. whether RED color is causing high rating or GREEN color. Its just the color.
If you OHE your data, then each value will become a feature and you may get the required significance value.

Can I claim that these above-mentioned attributes are good predictors of label 1 as its individual F1 is 75% but not for label 0, because it has only 30% F1 individually.

Yes, it should imply that i.e. splits on the feature separate out Label_1. Though I belive, a "Confusion matrix" is a better view for such analysis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.