4
$\begingroup$

If we have numeric variable, decision trees will use < and > comparisons as splitting criteria. Lets consider this case : If our target variable is 1 for even numeric value, and 0 for odd numeric value. How to deal with this type of variables? How to even identify these type of variables if we have large number of variables? Is there any specific names for these type of variables?

$\endgroup$
1
5
$\begingroup$

I would call this bad feature engineering, I'm afraid: as the designer of a ML system, one is supposed to analyze their data and find the best way to make the ML system perform as well as possible. In this case by adding a simple feature x % 2 for every instance the decision tree can perform perfectly.

[added] Even in the case of a more complex pattern, if there are such "clusters" of numerical values then there must be a logical explanation why this happens, i.e. something which depends on the task that an expert in this problem can analyze and understand. In most real cases this implies that there are some hidden/intermediate variables, and designing the system so that it represents these variables is key. In other words, the numeric variable is not directly semantically relevant for predicting the response variable, because the assumption when using numeric values is that their order matters (here the numeric value behaves more like a categorical variable).

$\endgroup$
7
  • $\begingroup$ As you said we will get the tree with single node if we have derived variable. But this(X%2=0) is not the logic every time. I think there should be a hack to identify these type of variables. Once we identify them we can try with different logics. My question is how to even identify these variables? Is there any specific names for these type of variables especially in the case of tree based algos? $\endgroup$ Apr 4 '20 at 12:59
  • 1
    $\begingroup$ @VenkateshGandi see updated answer $\endgroup$
    – Erwan
    Apr 4 '20 at 13:25
  • $\begingroup$ Is there a way to identify these type of variables in our analysis? $\endgroup$ Apr 4 '20 at 13:51
  • 2
    $\begingroup$ @VenkateshGandi it depends what is the actual "type" of variable: you can always run many kinds of mathematical tests (for example with modulo operator), but if you have no idea what you're dealing with it's not very efficient. At the end of the day the variable is supposed to have a semantic meaning for the task, and that's your best clue to know ho to use it. $\endgroup$
    – Erwan
    Apr 4 '20 at 14:40
  • $\begingroup$ Can we say by using visual inspection of tree we can identify these type of variables? (like the same variable used for consecutive splits.) $\endgroup$ Apr 4 '20 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.