If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : If our target variable is 1 for even numeric value, and 0 for odd numeric value. How to deal with this type of variables? How to even identify these type of variables if we have large number of variables? Is there any specific names for these type of variables?
$\begingroup$
$\endgroup$
1
asked Apr 3, 2020 at 13:33
-
3$\begingroup$ cross-posted at stats.stackexchange.com/q/458281/232706 $\endgroup$– Ben Reiniger ♦Apr 3, 2020 at 15:54
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
I would call this bad feature engineering, I'm afraid: as the designer of a ML system, one is supposed to analyze their data and find the best way to make the ML system perform as well as possible. In this case by adding a simple feature x % 2 for every instance the decision tree can perform perfectly.
[added] Even in the case of a more complex pattern, if there are such "clusters" of numerical values then there must be a logical explanation why this happens, i.e. something which depends on the task that an expert in this problem can analyze and understand. In most real cases this implies that there are some hidden/intermediate variables, and designing the system so that it represents these variables is key. In other words, the numeric variable is not directly semantically relevant for predicting the response variable, because the assumption when using numeric values is that their order matters (here the numeric value behaves more like a categorical variable).
-
$\begingroup$ As you said we will get the tree with single node if we have derived variable. But this(X%2=0) is not the logic every time. I think there should be a hack to identify these type of variables. Once we identify them we can try with different logics. My question is how to even identify these variables? Is there any specific names for these type of variables especially in the case of tree based algos? $\endgroup$ Apr 4, 2020 at 12:59
-
1
-
$\begingroup$ Is there a way to identify these type of variables in our analysis? $\endgroup$ Apr 4, 2020 at 13:51
-
2$\begingroup$ @VenkateshGandi it depends what is the actual "type" of variable: you can always run many kinds of mathematical tests (for example with modulo operator), but if you have no idea what you're dealing with it's not very efficient. At the end of the day the variable is supposed to have a semantic meaning for the task, and that's your best clue to know ho to use it. $\endgroup$– ErwanApr 4, 2020 at 14:40
-
$\begingroup$ Can we say by using visual inspection of tree we can identify these type of variables? (like the same variable used for consecutive splits.) $\endgroup$ Apr 4, 2020 at 16:25