In gradient descent, I know that local minima occur when the derivative of a function is zero, but when the loss function is used, the derivative is equal to zero only when the output and the predicted output are the same (according to the equation below).
So, when the predicted output equals the output, that means the global minima is reached! So, my question is: How can a local minima occur, if zero gradient occurs only for the "perfect" fit?
$$\theta_j := \theta_j - {\alpha \over m} \sum_{i=1}^M (\hat y^i-y^i)x_j^i$$