# Conjugated gradient method. What is an A-matrix in case of neural networks

I am reading about conjugated gradient methods to understand how they exactly work. I understand that a pair of vector $$u$$ and $$v$$ are conjugated with respect to $$A$$ if $$u^TAv=0$$. I also read that $$A$$ is symmetric, positive definite matrix.

I am trying to find out how is that related to training of neural network by minimalising mean-square error function using CG method. What will be $$A$$ matrix in that case? How $$A$$ matrix is connected to weights of neural network. And is it still symmetric and positive definite? I read What is conjugate gradient descent? this thread and resources linked there, but I still can't figure it out.

I'm sure I'm missing something simple, but could you give me a bit of explanation?

Thank you,

Max

The traditional conjugate gradient descent is an increment on the gradient descent that just takes a direction that is fully orthogonal to the previous descent direction. There is no $$A$$ matrix in that case.
• Simple scalar product. That's the basic definition (equivalent to $A=I$). Jan 2 '19 at 20:52