How to interpret fit from regression (decision) tree which has used 0 variables

Question

I have fit a regression tree to my dataset and the output from summary(tree1) is as follows:

Regression tree:
tree(formula = y ~ X)
Variables actually used in tree construction:
character(0)
Number of terminal nodes:  1 
Residual mean deviance:  0.985 = 71680 / 72770 
Distribution of residuals:
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-1.0050 -1.0050 -0.3122  0.0000  0.7842  2.2140 

So seemingly the tree has found no variables/splits worth including. The reason this is curious is because other methods have found trends in the data. For example a linear regression line found several significant variables and p-value for F-statistic was ~0 (although this is a very noisy dataset based on human behaviour and R-squared was not much bigger than 0).

How can I interpret this regression tree output? There are clearly trends in the dataset as shown by linear regression. Are regression trees more focused on accurate predictions and so won't fit well to a noisy dataset?

Answer 1

It is hard to tell what is going on without knowing the data and your actual approach/method/model/code.

Just some remarks. Linear regression is parametric and results hinge on the assumptions behind the parameterization. So maybe not too good in a noisy setup.

Single trees are often weak learners. The advantage of trees however is, that there is no parameterization behind. So they are in principle rather flexible. It is state of the art to combine many trees to one ensemble (random forest).

Especially with noisy data, boosting (also tree based) is really good. Boosting makes a lot of very small trees and (by updating weights to specific observations), it focusses on cases which are hard to predict. I guess this would be a thing to try if you work with predictive modeling.

Here are some very basic Python examples for boosting if you want to give it a try: https://github.com/Bixi81/Python-ml