How is the weight of each new weak learner is calculated in Xgboost?

Question

In Xgboost we have multiple sequential weak learner.

Let say I have weak learner WL1 and we fitted it on our data and we calulated the error. Now we have another weak learner WL2. And as I have read that now we will give more wieghtage to misclassified data. So can someone please explain me how to claculate how much weightage will be given to the misclassified data. and Please correct me if my understanding is wrong.

And can some tell me the fundamental difference between xgboost and Ada Boost.

Note: I have gone though 5-6 blog page and it confused me a lot. And no body has mentioned about the caluclation of the weight that is given to the new weak learner.

Can someone please explain a reliable source to understand the Xgboost and Adaboost in detail.

Answer 1

Your description sounds a bit more like AdaBoost than Gradient Boosting (XGBoost and others). AdaBoost trains its weak estimators on the original targets, just weighting/sampling the observations according to misclassification. In contrast, Gradient Boosting trains each weak learner on different targets, namely the "pseudo-residuals", i.e. the gradient (wrt the observations) of the objective function. (That has a similar effect, because misclassified observations will tend to have higher pseudoresiduals.)

See also:
Adaboost vs Gradient Boosting
CV: Is Gradiant Boosting a generalization of Adaboost?
CV: Intuitive explanations of differences between Gradient Boosting Trees (GBM) & Adaboost