In policy gradient, we have something like this:
Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ what are inputs of crossentropy? logits and action or logits and target? $\endgroup$– Kang_KaiFeb 23, 2019 at 9:20
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, just take the cross-entropy loss of the last layer and take the gradient with respect to it. The actual action will be the target.
For example, in PyTorch: apply CrossEntropyLoss on the last layer (no need of doing softmax, as it is done implicitly by this function)
