# Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones?

Is it possible to do hard-coded decision tree on some variables and random forest / something on the remaining ones?

The situation seems that for some variables it's possible to draw strong empirical assumptions, but for others their "relative importance" seems more random.

So e.g.

Researcher is certain that splitting X1 > 5 and X2 < 3 gives best information, since they are empirically sound splits e.g. based on stakeholder views. And X1, X2 are more important than X3, X4, X5, since X3, X4, X5 are redundant, if X1 or X2 don't exist.

Thus the model could essentially be based on X1, X2 only , but X3, X4, X5 should add explanatory power. Yet their relative importances are not known. Using the decision tree to them might be prone to model inaccuracies due to random forest or something perhaps offering better reduction in overfitting etc.

• Definetely, yes, the questions is how to do in pythonic way. Or whatever you are coding Jul 12 at 7:52
• @CarlosMougan I don't find it definite, because how to e.g. balance the metrics of the model, if part of it is decision tree and part is RF. Then what's the total accuracy for example? Jul 12 at 8:25
• @mavavilj performance (accuracy or other) can be calculated for any predicting system, including a human making predictions for example. It is not related to how the predicting system is made. As I said in your other question you probably have some confusion about the distinction between a predicting system and evaluation. Jul 13 at 20:13
• @Erwan I understand the technical difference, but I'm confused about the difference of accuracy gained by "expert opinion" and "computational methods", as in this example: pubmed.ncbi.nlm.nih.gov/9555627. So, if we have "strong expert opinion" on X1, X2 above, but not on X3, X4, X5. Then this could motivate one to use different methods on them. While the global prediction should still be aggregated from all. Using e.g. RF for all could distort expert opinion on X1, X2 in favor of "numerical idea of truth". Evaluation is also pointless, if the model is faulty(?) Jul 14 at 4:56
• @mavavilj this abstract doesn't give any detail about what the authors do. More importantly, selecting the features is like choosing a method to solve the problem, as you said. But evaluation is not about the method, it's about determining how well the method works. Evaluation can be applied to any system, whether it's good or bad, including a system which gives random predictions or always the same output. If the system is faulty, then evaluation should return a low performance score, that's it. So it must be possible to compare any system by their performance. Jul 14 at 9:45