# In Gradient descent, Why the gradient of cost function do not have to be normalized into unit vector

From my background, I understand that the purpose of having a learning rate (α) is to normalize the magnitude of gradient (▽J), so the step size can properly converge the local minima

Since α is arbitrary, so we have to do find the best hyperparameter learning rate (α).

Perhap, we do not have to hyper-parameter tuning when we normalize the gradient into a unit vector in which its magnitude always 1

w:=w −α▽J


w:=w −▽J/||▽J||


Again, back to my problem, I do not understand why the variant of gradient descent does not have what I think. Like normalize the gradient into unit vector

ps. I presume that my Gradient Descent might not work well when the magnitude of gradient is extremely low and could diverge to inf

The most obvious reason is that a gradient of the norm of 1 is expected to be at learning_rate * 1 away from the loss function minimum. That is averaged of course.