I'm doing a course on reinforcement learning, and one of our tasks is to implement an agent on the Lunar lander continuous V2 environment from openAI gym. In order to solve the continuous problem, I used discretization for the number of actions, where the number of actions is supposed to be at least 8 (I chose 10). I used 3 methods: Deep Q Network, Double Deep Q Network and dueling, and I tried to solve it in two configurations: one regular and one with random noise (adding a zero-mean Gaussian noise with mean=0 and std = 0.05 to PositionX and PositionY observation of the location of the lander.)

The results I get: without noise: DQN reached avg. of 234, DDQN reached avg. of 229 and 3DQN reached avg. of 256. We can see that the 3DQN reaches much higher values in most episodes and the lowest values are still above 40. with noise: Total rewards per episode for DQN, DDQN, 3DQN train with random noise. We can see convergence at around 500 episodes for DDQN and 3DQN and around 800 episodes. The 3DQN is very stable and reaches high values faster than the DQN and DDQN.

Are these results reasonable? What are the main differences between the models that can explain these results? We also found that a network with about 2-3 hidden layers performs better than a deeper layer network. Does that makes sense?

  • $\begingroup$ Anyone? I'll even settle for a deep insights about the differences between the various models, and then I'll try to figure out myself why model x performs better in situation y. Thanks! $\endgroup$
    – user113367
    Commented Mar 16, 2021 at 5:01


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.