In policy gradient, we have something like this:
Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this communityIn policy gradient, we have something like this:
Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
Yes, just take the cross-entropy loss of the last layer and take the gradient with respect to it. The actual action will be the target.
For example, in PyTorch: apply CrossEntropyLoss on the last layer (no need of doing softmax, as it is done implicitly by this function)