Could someone explain why the target of the DDPG's policy is $Q(s,\mu(s))$?
My understanding of DDPG is like this. Since it is intractable to calculate the $argmax_a Q(s,a)$ in a continuous space, DDPG uses an universal function estimator (Neural Network) to learn and predict the best action that realize $maxQ(s,a)$ output.
So, my question is that what is the actual target when DDPG trains $\mu(s)$? I thought that it should be an actual action that gives the highest Q-value at given state $s$ ($argmax_aQ(s,a)$). However, in OpenAI spinning up, it says it can make approximation that $max_aQ(s,a)\approx Q(s,\mu(s))$, and says the loss of the policy is $E[Q(s, \mu(s))]$.
What does this mean? Are they targeting the Q-value, instead of action-value? How can this policy can learn the best action?