We normally use the online network for action selection and the target network for evaluation , would there be a difference if we switched the roles?
Because in the case Of Double Q learning, we alternate between the two estimators, this isn’t the case in Double deep Q learning
If our target network is the one that actually overestimated it’s values how do we correct that
Eg Q(s,a) =R + y Q(s’, argmax(Q*(s’,a,w-)),w)
In this example our online network has weights w And our target network has weights w- As you can see the target network performs action selection And our online network performs evaluation
Would there be any issue with doing it this way?