Could you please tell me why do we use a learning rate to move into the direction of the derivative to find the minimum? Why is it not good if you simply count it where is it 0?
Learning rate gives the rate of speed where the gradient moves during gradient descent. Setting it too high would make your path instable, too low would make convergence slow. Put it to zero means your model isn't learning anything from the gradients.
Usually cost function used in gradient descent are convex as should in image above. This will be similar also for data with multiple features because for such data we can reason this in similar way one feature at a time.
Let's say we are at point A during training at which point gradient is G, which means cost is increasing fastest in the direction in the direction of G. So we want to move in opposite direction of G with some step size which is learning rate.
As in fig above cost is increasing in direction of negative $w_2$ axis so we want to move in the direction of positive $w_2$ axis. But if we move too much in that direction i.e at point $w_2=6$ then actually cost value has increased. If we always move in same rate then we'll never reach the minimum point.
So we'll need a learning rate which is suitable for this cost function so that it is large enough that we'll have fast descent but low enough that it doesn't shoot other side of the curve
Let's take the function Y = x^2. Where is its minimum? It's where dy/dx = 0. Let's find that. dy/dx = 2x. But why 2x? Or simply how do we find a derivative function for any function just that easy? It's because we exactly know the relationship between Y and X. That is Y is always X*X. And same for all functions. Let's come to some data. Do we know the exact relationship between the input X and the target Y? No. Because we can't say 'Y is always "this" relationship with X' and that is actually what we r trying to find out. So what can we do ? Take small baby step in a way that our error is minimizing. Making big move might lead us to bad place.
So to answer ur question, in a case of Y = x^2, we know the relationship, so we find the minimum by formula not by analysis. But in case where the problem is mapping input data X to target Y, (where we don't know the relationship), we can't use formula to get minimum but analysis. U can use analysis for the former case, but it's little tedious I guess.