I saw in DQN nature paper 2015 https://www.nature.com/articles/nature14236(Extended Data Table 4) some comparisons between DQN and linear Q-learning. Extended Data Table 4

The ratio of reward for different games are very different. For instance I saw the ratio of reward of Linear Q-learning is between 3.8 to 100 times more than DQN? What is the reason of this big difference? How we can clarify this? Is it related to complexity or sensitivity of game? I am not very familiar with complexity and properties of these games.

In addition, in Extended Data Table 2 of this paper it has written it is compared with best linear learner?

Would you please explained this sentence:"Best Linear Learner is the best result obtained by a linear function approximator on different types of hand designed feature"

What is the meaning that by best? The best results during test of linear? Because I only know one linear regression


, I only replace the DNN section of DQN with a linear regression (i.e. just input and output and linear without any hidden layer) I am confused with best word in this table. Extended Data Table 2

Also, do you have seen the comparison with Logistic Regression Q-learning? (What I mean by logistic regression Q-learning is to repalce the deep neural network section of DQN with a Sigmoid function between input and output without any hidden layer) What is the ratio of result of this approach in comparison with DQN?

  • 1
    $\begingroup$ Q-learning involves a regression, not a classification, so logisitic regression wouldn't really make sense. Logistic regression would be the baseline for policy learning. Good question though about what the linear learner is! $\endgroup$
    – dashnick
    Oct 15, 2018 at 12:12
  • 1
    $\begingroup$ That's what I mean - the q function should return a real number between + and - infinity; logistic regression is squashed between 0 and 1 $\endgroup$
    – dashnick
    Oct 15, 2018 at 12:53
  • $\begingroup$ @neil-slater do you have any suggestion? $\endgroup$ Oct 19, 2018 at 17:23
  • $\begingroup$ @ Neil Slater do you have any suggestion? $\endgroup$ Oct 19, 2018 at 17:24


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.