Let's scope this to just classification.

It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy down to Bayes error*.

Is this universally true for all non-parametric methods?

*Assuming the model has access to the "right" features.

  • $\begingroup$ there is no known mathematical result that applies to "non-parametric models" in general. However it seems to be the case $\endgroup$ – Nikos M. Jul 16 at 12:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.