Does gradient descent always find global minimum for specific regression type?

Question

From my understanding, linear regression is used for predicting an output based on an input using a linear equation that is optimally fitted to some input data. We choose the best fitted linear equation for some input data using a loss function. By simulating the values of m and b in y = mx + b we can find the optimal linear equation with gradient descent.

My question is, does gradient descent always find the global minimum loss for linear regression? An extension of this question would be, doesn't the answer to the previous question depend on the loss function used? Furthermore, when we use gradient descent on a plot of m, b, and the value of our loss function, is the plot always convex given that we are using linear regression?

Answer 1

For linear regression, using least square error as the loss function, the cost function will be convex. Gradient descent applied to a convex function will converge to a global minimum. And yes, it depends on the loss function. I don't have an example of a non convex loss function for linear regression but check this video for the case of logistic regression.