Decision trees are often used in machine learning as classifiers/regression models (CART), or in ensemble methods (Random Forest) etc, where predictive accuracy, minimizing bias and variance are the priority, and having a complicated tree is not necessarily a bad thing.
My application is different. I have run a simulation ~5000 times with different parameter combinations (chosen from a Latin Hypercube), and I extract response variables (in the range [0,1] where 1 = good) from each simulation. I want to identify which parameter ranges lead to "good results" in the response variables in such a way as can be easily interpreted by a non-technical user. Pairwise correlation between factors and responses identifies relatedness and direction, but not significance and critical values.
So, I can create a decision tree for each response variable, from the parameters, which partitions the parameter space into the ranges which help predict the response. Identifying the "best leaf" (max average leaf value) I can extract the ranges and parameters along the path to the root and convert this to a simple explanation like:
"ResponseX attains average 0.999 when Factor1 >= 10, Factor6 <= 4 and Factor9 >= 3"
However what I find is that the decision trees produced (with scikit-learn) are often highly complex and sometimes make miniscule distinctions in response values (eg. splitting on 0.811 and 0.812). I want to identify the significant relationships only which should be a relatively small number.
So unlike in ML, I want to reduce complexity of the tree instead of maximizing predictive accuracy of a regression model.
How can I do this? How is it typically done?
I see parameters like max_depth and max_features and I found that setting min_impurity_decrease helped a bit, but I would like to know the convention way.