2
$\begingroup$

For a binary classification problem, I have a small data set with 200 observations. There are around 20 potential variables but based on variance importance I think only 2 or 3 are important for classification. This data set is too small to train a random forest model for prediction purposes, but is it okay to run a random forest model and use the variable importance feature to understand which variables are important? If only a handful of features are by far the most important, then, I think despite the small data size, it's an appropriate approach in that it'll tell me what I want. The results can then be used to perhaps build a simple decision tree or as a preliminary analysis for a future larger test.

Any flaws with this thinking?

$\endgroup$
  • $\begingroup$ There is absolutely no problem with this method. I would also suggest training a simple decision tree model and observe the tree itself, it can be very useful when working with a small number of variables. $\endgroup$ – Erwan Jun 24 at 11:45
2
$\begingroup$

A Random Forest model can definitely be used to help you determine feature importances. Actually, it is used as a very common strategy for feature selection. If your data is too small, my recommendation would be to treat it as if you were to make predictions with this model, meaning that you should watch out for overfitting and do a proper hyperparameter optimization (you want a simpler and smaller model here), so that the features you find are really the most relevant, and don't just appear important for being biased by the small training set.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.