Why use Gradient Descent when Gradient just solve the problem? (With Neural Nets)

My knowledge is that gradient actually gets to the global minimum and gradient descent try to take steps to the direction that he judges to be the lowest. I know that calculate the gradient from complex functions is non trivial problem, but let's say that we are a using neural net with a quadratic cost function, would be that hard to calculate the gradient and get actual global minimum?