Timeline for Policy gradient/REINFORCE algorithm with RNN: why does this converge with SGM but not Adam?
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 11, 2019 at 21:09 | comment | added | Kechen | so you think the problem is the implementation? Do you have any experience on training RNN with RL objective? | |
| Apr 11, 2019 at 20:37 | comment | added | Ran Elgiser | @Kechen But that shouldn't make the model diverge, It should only make it learn slower | |
| Apr 11, 2019 at 20:14 | comment | added | Kechen | I do start from a very small learning rate, but it does not help. I am wondering if adam failed because of its momentum mechanism. In RL sequence training, it is very likely to sample a very bad sequence which has 0 reward and the model get 0 gradient with that sample. However, adam with momentum still updates the model even if the gradient is zero. Would that be a problem? | |
| Apr 10, 2019 at 21:25 | review | First posts | |||
| Apr 10, 2019 at 21:37 | |||||
| Apr 10, 2019 at 21:20 | history | answered | Ran Elgiser | CC BY-SA 4.0 |