# What exactly is Gradient norm?

I found that there is no common resource and well defined definition for "Gradient norm", most search results are based on ML experts providing answers which involves gradient norm or papers which reference it and provide a single sentence intro to it.

Is there any well defined resource I can refer to get a concrete understanding of it ? Thank you

The concept of norm comes from functional analysis (and is found both in linear algebra and in optimization methods and other fields). There are several types of norm. The one in question is the norm of Euclidean space (which is the square root of the rocky product of vectors).

(a,a)^(1/2) = ||a|| (a,a) = a_1a_1+...+a_na_n when a = (a_1, ..., a_n)

The gradient is the point of the greatest growth of the function at the selected point (in fact, it is a vector of partial derivatives of the function from n variables).

] f(x,y) --> grad u = ( f′x(x,y), f′y(x,y))

So the gradient norm is the rock product of the vector of partial derivatives by the vector of partial derivatives.