In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions?

0- What is the meaning of this sentences "Each agent was evaluated every 250,000 training frames for 135,000 validation frames" in DQN nature paper?

2- What about meaning of this sentences: "Note that these evaluation episodes were not truncated at 5min leading to higher scores on Enduro"

1- Does this correct:we just put a network with an input and an output? I do not sense why the performance of it is very how?

enter image description hereTheperformanceoftheDQNagentiscomparedwiththeperformanceofalinearfunctionapproximator on the 5 validation games (that is, where a single linear layer was used instead of the convolutional network, in combination with replay and separate target network). Agents were trained for 10 million frames using standard hyper parameters,and three different learning rates. Each agent was evaluated every 250,000 training frames for 135,000 validation frames and the highest average episode score is reported. Note that these evaluation episodes were not truncated at 5 min leading to higher scores on Enduro than the one s reported in Extended Data Table2. Note so that the number of training frames was shorter (10 million frames) as compared to the main results presented in Extended Data Table 2 (50 million frames).

  • $\begingroup$ A linear function means $y=ax+b$, which is the same definition as everywhere else. In special case as an activation function, they often mean the simplest linear function $y=x$. ReLU, softmax, sigmoid are all not linear. (However ReLU is piece-wise linear.) $\endgroup$
    – user12075
    Oct 2, 2018 at 14:19

1 Answer 1


0: After 250,000 steps in the training loop they start a validation environment to check the current performance without fitting the weights of the current neural networks.

2: One episode of a validation episode just takes at most 135,00 frames (steps) and were not truncated by time.

1: Some games are more difficult to learn than others and the DQN approach is more suitable to some games than to others.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.