Is it appropriate to use random forest not for prediction but to only gain insights on variable importance?

Question

For a binary classification problem, I have a small data set with 200 observations. There are around 20 potential variables but based on variance importance I think only 2 or 3 are important for classification. This data set is too small to train a random forest model for prediction purposes, but is it okay to run a random forest model and use the variable importance feature to understand which variables are important? If only a handful of features are by far the most important, then, I think despite the small data size, it's an appropriate approach in that it'll tell me what I want. The results can then be used to perhaps build a simple decision tree or as a preliminary analysis for a future larger test.

Any flaws with this thinking?

Answer 1

A Random Forest model can definitely be used to help you determine feature importances. Actually, it is used as a very common strategy for feature selection. If your data is too small, my recommendation would be to treat it as if you were to make predictions with this model, meaning that you should watch out for overfitting and do a proper hyperparameter optimization (you want a simpler and smaller model here), so that the features you find are really the most relevant, and don't just appear important for being biased by the small training set.