0
$\begingroup$

Decision trees are often used in machine learning as classifiers/regression models (CART), or in ensemble methods (Random Forest) etc, where predictive accuracy, minimizing bias and variance are the priority, and having a complicated tree is not necessarily a bad thing.

My application is different. I have run a simulation ~5000 times with different parameter combinations (chosen from a Latin Hypercube), and I extract response variables (in the range [0,1] where 1 = good) from each simulation. I want to identify which parameter ranges lead to "good results" in the response variables in such a way as can be easily interpreted by a non-technical user. Pairwise correlation between factors and responses identifies relatedness and direction, but not significance and critical values.

So, I can create a decision tree for each response variable, from the parameters, which partitions the parameter space into the ranges which help predict the response. Identifying the "best leaf" (max average leaf value) I can extract the ranges and parameters along the path to the root and convert this to a simple explanation like:

"ResponseX attains average 0.999 when Factor1 >= 10, Factor6 <= 4 and Factor9 >= 3"

However what I find is that the decision trees produced (with scikit-learn) are often highly complex and sometimes make miniscule distinctions in response values (eg. splitting on 0.811 and 0.812). I want to identify the significant relationships only which should be a relatively small number.

So unlike in ML, I want to reduce complexity of the tree instead of maximizing predictive accuracy of a regression model.

How can I do this? How is it typically done?

I see parameters like max_depth and max_features and I found that setting min_impurity_decrease helped a bit, but I would like to know the convention way.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.