# XGBoost equations (for dummies)

I am having a hard time trying to understand the MSE loss function given in the Introduction to Boosted Trees (beware! My maths skills are the equivalent of a very sparse matrix):

$\begin{split}\text{obj}^{(t)} & = \sum_{i=1}^n (y_i - (\hat{y}_i^{(t-1)} + f_t(x_i)))^2 + \sum_{i=1}^t\Omega(f_i) \\ & = \sum_{i=1}^n [2(\hat{y}_i^{(t-1)} - y_i)f_t(x_i) + f_t(x_i)^2] + \Omega(f_t) + constant \end{split}$

The second equality sign implies that one could easily derive the second equation from the first one, but I cannot see how. My first naïve attempt was to:

• express $y_i$ as $a$
• express $(\hat{y}_i^{(t-1)} + f_t(x_i))$ as $b$
• and then expand $(a-b)^2$

But I wasn't successful. Any help is really appreciated.

I recall I was struggling for some time deriving the second equation. That constant keeps many of your missing elements. Let's break it down using your $(a-b)^{2}$ notation. We will have $a^{2}$, $b^{2}$, and $2ab$:
• $a^{2}$: $y_{i}$ is constant since it is your true labels/values, thus $a^{2}$ i.e. $y_{i}^{2}$ goes to the constant.
• $b^{2}$: $y\hat{}_{i}^{t-1}$+$f_{t}^{2}$+$2y\hat{}_{i}^{t-1}f_{t}$
$y\hat{}_{i}^{t-1}$ is constant as it is prediction in a step before $(t-1)$ that we know already, thus goes to the constant term. The other two terms remain as it is.
• $2ab$: $2y_{i}(y\hat{}_{i}^{t-1}+f_{t}) = -2y_{i}y\hat{}_{i}^{t-1}-2y_{i}f_{t}$. Here also the first term is constant. Only the second term remains.