I am testing different RL methods, and I know e.g that policy gradient method is supposed to have a high variance gradient which can cause trouble. I want to run a few different Deep RL algorithms, and see if I can see any patterns in the gradient. How can I visualize the gradient, and analyze it?

So far I have been using the average standard deviation, of the gradient, for every layer in my neural network and visualized it as a scalar with epoch on the x-axis. Is there a better way?

  • $\begingroup$ Could you elaborate a bit more on what you expect to see from the gradients? I am curious as you seem that you are searching for something beyond simple statistics of the samples that estimate the gradients. $\endgroup$ Oct 18, 2018 at 23:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.