I wonder if decision trees (and their derivatives like Random Forest and Gradient Boosting) have interpolation power as deep learning based model. Most of my experience is with deep learning model.

Correct me if I am wrong, but unlike neural network, decision tree works with simple threshold. This means that the impact of each feature is binary decision - does the feature is above or below the threshold. In the case of a regression task, this means the output of the decision tree model will result in a non-smooth step function (although smoothing can be approximating by many steps or ensemble of trees).

Neural Network (NN) on the other hand, can yield a smooth output function. Unlike NN ,and again, please correct me if I am wrong, a perfect linear regression function cannot be achieved by a finite set of decision tree.

I don't have a lot of experience in using decision tree (specifically gradient boost).



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.