difference between model-based boosting and gradient boosting

Question

What exactly is the difference between model-based boosting and gradient boosting? For an intro to model-based boosting see https://cran.r-project.org/web/packages/mboost/vignettes/mboost_tutorial.pdf It seems to me that both terms are equivalent. However, both are used in various literature...

Answer 1

Gradient Boosting is fitting a base learner $f_{i}(X)$ to the gradient of the loss function of an existing model $F_{i-1}(X)$ i.e. find base learner $f_i$ which minimises $L(-g_i, f_t(x_i))$ where $g_i$ is the gradient of $L(y_i,\hat{y}_i)$ with respect to $\hat{y}=F_{i-1}(X)$ at the current iteration $i$. Effectively it's gradient descent in function space.

Component wise boosting schemes such as that used by mboost have a list of base learners of which one is selected at each step, i.e.

form2 <- y ~ bols(x1) + bols(x2) + bols(x1, by = x2, intercept = FALSE) +
               bspatial(x1, x2, knots = 12, center = TRUE, df = 1)

Specifies 4 possible base learners, bols(x1), bols(x2), bols(x1,by=x2) and bspatial(x1,x2), all of which are regression splines.

More generally gradient boosted decision trees fits a tree at each step. So the base learners are arguably more complex.

I believe the terms model based and functional are equivalent and both `mboost' and GBDT are examples.

Answer 2

I think model-based boosting is a generalization of gradient boosting which allows for more complex base predictors then trees.

Reference: http://www.jmlr.org/papers/volume11/hothorn10a/hothorn10a.pdf