I have a classification problem. In gradient tree boosting I read that-
1. Initially a weak learner is fitted on the entire training dataset.
2. Output of each training row is obtained. In my case it will be {0,1}. 3. Now, the second classifier will train on the residual of the prediction i.e {initial prediction - final prediction}.
My doubt is it seems very odd to calculate the residual in case of classification problems while it is ok when I look at residual calculation in case of mse.
So, what is actually happening in case of categorical target variable. How is the residual being calculated and which loss function is that fitted into?

Thanks in advance.


1 Answer 1


My layman's understanding is that binary classification is usually calculated using the logit transform.

I believe then that the residuals are the difference between the response and the predicted probabilities and the default metric for this application is the Area-Under-Curve (AUC).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.