4
$\begingroup$

The major limitations of decision tree approaches to data analysis that I know of are:

  1. Provide less information on the relationship between the predictors and the response.
  2. Biased toward predictors with more variance or levels.
  3. Can have issues with highly collinear predictors.
  4. Can have poor prediction accuracy for responses with low sample sizes.

Are there any others? Are they robust to traditional statistical assumptions such as homogeneity, normality, independence?

$\endgroup$
4
$\begingroup$

Simple decision trees have some limitations listed below. Fortunately, some of these can be fixed used ensemble learning techniques (think bagging, boosting...).

Concerning limitations :

  • Trees tend to overfit quickly at the bottom. If you have few observations in last nodes, poor decision can be taken. In this situation, consider reducing the number of levels of your tree or using pruning.

  • Trees can be unstable because small variations in the data might result in a completely different tree being generated.

  • Decision trees perform greedy search of best splits at each node. This is particularly true for CART based implementation which tests all possible splits. For a continuous variable, this represents 2^(n-1) - 1 possible splits with n the number of observations in current node.

  • For classification, if some classes dominate, it can create biased trees. It is therefore recommended to balance the dataset prior to fitting.

    Also, Some distributions can be hard to learn for a decision tree. An example below (XOR) :

enter image description here

| improve this answer | |
$\endgroup$
3
$\begingroup$

1)Over Fitting is one of the most practical difficulty for decision tree models. This problem gets solved by setting constraints on model parameters and pruning.

2)Not fit for continuous variables: While working with continuous numerical variables, decision tree looses information when it categorizes variables in different categories.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.