0
$\begingroup$

i would like to understand a bit more the mathematics behind xgboost. I understand that the hessian is the second partial derivative of the loss function, i originally thought this was with respect to the log of the odds. But what i do not understand is how it is a matrix, and how it applies to a matrix of training data where columns are feature and rows and number of samples.

https://www.youtube.com/watch?v=ZVFeW798-2I&t=126s i have watchrd this helpful video but this is looking at just one feature 'dosage'. if i have say many features how do the loss functions change. i am trying to view the bigger picture here but am struggling to see how we build a xgboost tree from matrix. below shows the hessian calculation.

enter image description here

is the loss function calcaulted per row? per feature? please help on maths of this , greatly appreciated! how is it calcaulted per row, since a row is a combination of features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.