I read that decision trees (I am using scikit-learn's classifier) are robust to outlier. Does that mean that I will not have any side-effect if I choose not to remove my outliers?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ Outliers in input or output? $\endgroup$– Michael MCommented Aug 24, 2018 at 21:07
-
$\begingroup$ Outliers in input. $\endgroup$– JishanCommented Aug 24, 2018 at 23:05
-
1$\begingroup$ Please edit your question to define what you mean by "outlier", and in what kind of robustness you are looking for. (This is a broad topic and there are multiple things you could mean.) $\endgroup$– D.W.Commented Aug 27, 2018 at 5:10
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
Yes. Because decision trees divide items by lines, so it does not difference how far is a point from lines.
$\begingroup$
$\endgroup$
4
Most likely outliers will have a negligible effect because the nodes are determined based on the sample proportions in each split region (and not on their absolute values).
However, different implementations to choose split points of continuous variables exist. Some consider all possible split points, others percentiles. But, in some poorly chosen cases (e.g. dividing the range between min and max in equidistant split points), outliers might lead to sub-optimal split points. But you shouldn't encounter these scenarios in popular implementations.
-
$\begingroup$ Can you please elaborate on
based on the sample proportions in each split region.What do you mean by this ? $\endgroup$ Commented Mar 18, 2019 at 5:06 -
$\begingroup$ after a node splits the data into two groups you only need to know the labels (or their counts) in each group to determine how good the split is (the outlier in the input variable has no influence on the calculation) $\endgroup$– oW_Commented Mar 18, 2019 at 15:14
-
$\begingroup$ @But if the technique used for splitting is variance reduction then while calculating the variance/standard deviation of two splitted groups(child nodes) vs the non-splitted group(parent node) outlier will have influence $\endgroup$ Commented Mar 19, 2019 at 11:07
-
$\begingroup$ Variance reduction is used for regression trees and, again, the variance refers to the variance in the target variable, not the inputs. Also the question was about classification. $\endgroup$– oW_Commented Mar 19, 2019 at 14:58
$\begingroup$
$\endgroup$
1
It actually depends on the criterion by which the nodes of the tree are split. If the criterion is sensitive to outliers (for example variance if used in a regression problem) this can cause problems.
On the whole though, they are quite robust.
-
$\begingroup$ Well if we talk about outliers in the inputs, that is not the case. If outliers exists in the target variable, yes, you are right. $\endgroup$– ARATCommented May 28, 2020 at 23:50
$\begingroup$
$\endgroup$
Yes all tree algorithms are robust to outliers. Tree algorithms split the data points on the basis of same value and so value of outlier won't affect that much to the split.
For example: Want to determine the buying behavior of customers depending upon their house size. House size is numeric continuous variable ranging from 1-1000 sq ft.
So , now consider majority of my customers house size in range of 100-500. If I got some customers with house size of 1000 then what it does is simply split the data on the basis of some value where entropy at next level is less than that of the current level.
Split gets decided on the value of house size such that i will get more homogeneous group of customers.
answered Oct 11, 2018 at 13:59