3
$\begingroup$

I read that decision trees (I am using scikit-learn's classifier) are robust to outlier. Does that mean that I will not have any side-effect if I choose not to remove my outliers?

$\endgroup$
  • $\begingroup$ Outliers in input or output? $\endgroup$ – Michael M Aug 24 '18 at 21:07
  • $\begingroup$ Outliers in input. $\endgroup$ – Jishan Aug 24 '18 at 23:05
  • 1
    $\begingroup$ Please edit your question to define what you mean by "outlier", and in what kind of robustness you are looking for. (This is a broad topic and there are multiple things you could mean.) $\endgroup$ – D.W. Aug 27 '18 at 5:10
4
$\begingroup$

Yes. Because decision trees divide items by lines, so it does not difference how far is a point from lines.

enter image description here

$\endgroup$
5
$\begingroup$

Most likely outliers will have a negligible effect because the nodes are determined based on the sample proportions in each split region (and not on their absolute values).

However, different implementations to choose split points of continuous variables exist. Some consider all possible split points, others percentiles. But, in some poorly chosen cases (e.g. dividing the range between min and max in equidistant split points), outliers might lead to sub-optimal split points. But you shouldn't encounter these scenarios in popular implementations.

$\endgroup$
  • $\begingroup$ Can you please elaborate on based on the sample proportions in each split region. What do you mean by this ? $\endgroup$ – Rookie_123 Mar 18 '19 at 5:06
  • $\begingroup$ after a node splits the data into two groups you only need to know the labels (or their counts) in each group to determine how good the split is (the outlier in the input variable has no influence on the calculation) $\endgroup$ – oW_ Mar 18 '19 at 15:14
  • $\begingroup$ @But if the technique used for splitting is variance reduction then while calculating the variance/standard deviation of two splitted groups(child nodes) vs the non-splitted group(parent node) outlier will have influence $\endgroup$ – Rookie_123 Mar 19 '19 at 11:07
  • $\begingroup$ Variance reduction is used for regression trees and, again, the variance refers to the variance in the target variable, not the inputs. Also the question was about classification. $\endgroup$ – oW_ Mar 19 '19 at 14:58
0
$\begingroup$

It actually depends on the criterion by which the nodes of the tree are split. If the criterion is sensitive to outliers (for example variance if used in a regression problem) this can cause problems.

On the whole though, they are quite robust.

$\endgroup$
0
$\begingroup$

Yes all tree algorithms are robust to outliers. Tree algorithms split the data points on the basis of same value and so value of outlier won't affect that much to the split.

For example: Want to determine the buying behavior of customers depending upon their house size. House size is numeric continuous variable ranging from 1-1000 sq ft.

So , now consider majority of my customers house size in range of 100-500. If I got some customers with house size of 1000 then what it does is simply split the data on the basis of some value where entropy at next level is less than that of the current level.

Split gets decided on the value of house size such that i will get more homogeneous group of customers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.