I am reading the original gradient boosting machine article and, maybe because my statistics are a bit rusty, have a few questions on one section.

In section 3. Finite Data the following claim is made about the previously introduced nonparametric approach to function approximation.

This nonparametric approach breaks down when the joint distribution of $(y, \bf{x})$ is estimated by a finite data sample $\{y_i, \bf{x_i}\}^N_1$. In this case $E_y[\cdot|\bf{x}]$ cannot be estimated accurately by its data value at each $\bf{x_i}$ ...

I am at a complete lack of understanding for this claim and justification, again perhaps because I am missing some stats. I have the following questions:

  1. What specifically about the finite data is causing this?
  2. Why is the nonparametric function approximation effected by this and not the parametric form


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.