I am new to reinforcement learning so I apologize for the wrong use of terms, if any. In SARSA, the value of a state-action pair is updated after the robot takes an action following its internal policy to move into a new state. The policy(action) for each state is then updated at the end of each episode.
In deep Q-Learning, the goal is now to estimate the value of each action from the current state. The input is thus a one-hot vector whose dimension is equal to the number of states and the output a vector containing the scores for each action.
If my explanation is correct, then my question is, how do we train the deep net? How do we get the ground truths? The robot can only, at each episode, move in one direction from one state. That would mean that the vector containing the scores for the action is zero everywhere except for one entry...?