I would like to clip the reward in keras. I saw it is possible to clip the norm and clip the value is sgd as follows:

sgd = optimizers.SGD(lr=0.01, clipnorm=1.)
sgd = optimizers.SGD(lr=0.01, clipvalue=0.5)

What are clipping the norm and clipping the value?

Also, How it is possible to implement the clipping the reward for Adam? Would you please let me know how I can do this?


Gradient clipping takes two main forms in Keras: gradient norm scaling (clipnorm) and gradient value clipping (clipvalue).

1. Gradient Norm Scaling

Gradient norm scaling involves changing the derivatives of the loss function to have a given vector norm when the L2 vector norm (sum of the squared values) of the gradient vector exceeds a threshold value. For example, we could specify a norm of 1.0, meaning that if the vector norm for a gradient exceeds 1.0, then the values in the vector will be rescaled so that the norm of the vector equals 1.0.

2. Gradient Value Clipping

Gradient value clipping involves clipping the derivatives of the loss function to have a given value if a gradient value is less than a negative threshold or more than the positive threshold. For example, we could specify a norm of 0.5, meaning that if a gradient value was less than -0.5, it is set to -0.5 and if it is more than 0.5, then it will be set to 0.5.

3. Keras Syntax

The gradient clipping syntax for Adaptive Moment Estimation (Adam) is very simple and follows the same syntax as for Stochastic Gradient Descent (SGD) shown above:

opt_adam = optimizers.adam(clipnorm=1.)
opt_adam = optimizers.adam(clipvalue=0.5)
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.