# What is purpose of partial derivatives in loss calculation (linear regression)?

I am studying ML and data science stuff from scratch. As a part of the course, I am studying how the models are derived. And for most of them, starting with the simplest - linear regression, we take partial derivatives. I understood its implementation part, however, I am a bit confused about why we need to take partial derivative there.

Is there any specific reason behind it? Can we use any other methodology to compute linear regression loss function?

The underlying idea behind machine learning is to come up with more or less complicated algorithms such that, given a set of input data, one is able to produce some sort of output; this output in turn depends on some parameters (upon which the model is specified). The objective is to choose the aforementioned parameters so that the algorithm output is as close as possible to the actual result. Now let $$y_i$$ and $$f(x_i, \beta)$$ be the actual and the predicted value, respectively, (in correspondence of the variable $$x_i$$): the previous sentence translates into trying to choose the parameters $$\beta$$ (whatever they mean) such that the error you commit is the smallest, namely $$L(y | x, \beta)$$ is minimised, where the function $$L$$ is any way we decide to estimate the error between predictions and actuals. In the literature $$L$$ is referred to as loss function and for most practical purposes, especially for polynomial models like linear regression, it reduces to the sum of squares $$L(y|x) = \sum_{i=1}^N (y_i -f(x_i,\beta))^2.$$ For each choice of parameters $$\beta$$ and function $$f(x_i)$$ the above takes different values; we are looking for, once we decide to fix the form of $$f$$, the set of $$\beta$$ such that the above is the smallest. Assuming the loss function is differentiable, local minima must satisfy the condition such that the collection of partial derivatives with respect to the variable in question (in this case $$\beta$$) must vanish; as such, at the end of the day one comes down to essentially taking derivatives and equating them to zero.
In case of linear regression one assumes $$f(x_i, \beta) = \sum_{j=1}^M x_i^j \beta_j$$: plugging this expression into the loss function and taking derivatives gives back the familiar expressions for the coefficients that one learns in school. Likewise for more complicated models: the form of the function $$f$$ might be more complicated, there might be analytical problems related to the computational minimisation of the loss, there might be a bunch of more parameters connected to each other in a somewhat complex way (for instance for neural networks), nevertheless the underlying argument still holds.