# What is conjugate gradient descent?

What is Conjugate Gradient Descent of Neural Network? How is it different from Gradient Descent technique?

I came across a resource, but was unable to understand the difference between the two methods. It has mentioned in the procedure that:

the next search direction is determined so that it is conjugate to previous search directions.

What does this mean? Also, what is line search mentioned in the web page?

Can anyone please explain it with the help of a diagram?

• This paper, while long, builds up the CGD from simple linear algebra. – Jérémie Clos Sep 30 '15 at 13:11
• The wikipedia article actually does a pretty good job of illustrating the difference between conjugate gradient method and the gradient descent method and even approaches conjugate gradient from the perspective of both a direct solve and an iterative solve. If you don't like math, then Andrew Ng does a pretty good job of describing both with essentially no math in his Coursera Machine Learning Course – AN6U5 Sep 30 '15 at 16:36
• If I have a curve, like a bell say, then how will one apply conjugate gradient? I went through the wiki, did not understand. I wanted a pictorial description – girl101 Oct 1 '15 at 3:37

$\mathrm {x^T A y} = 0$ instead of $\mathrm{x^T y} = 0$