i would like to understand a bit more the mathematics behind xgboost. I understand that the hessian is the second partial derivative of the loss function, i originally thought this was with respect to the log of the odds. But what i do not understand is how it is a matrix, and how it applies to a matrix of training data where columns are feature and rows and number of samples.

https://www.youtube.com/watch?v=ZVFeW798-2I&t=126s i have watchrd this helpful video but this is looking at just one feature 'dosage'. if i have say many features how do the loss functions change. i am trying to view the bigger picture here but am struggling to see how we build a xgboost tree from matrix. below shows the hessian calculation.

enter image description here

is the loss function calcaulted per row? per feature? please help on maths of this , greatly appreciated! how is it calcaulted per row, since a row is a combination of features.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.