In policy gradient, we have something like this: enter image description here

Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?

  • $\begingroup$ what are inputs of crossentropy? logits and action or logits and target? $\endgroup$
    – Kang_Kai
    Feb 23, 2019 at 9:20

1 Answer 1


Yes, just take the cross-entropy loss of the last layer and take the gradient with respect to it. The actual action will be the target.

For example, in PyTorch: apply CrossEntropyLoss on the last layer (no need of doing softmax, as it is done implicitly by this function)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.