# How to implement clipping the reward in DQN in keras

How to implement clipping the reward in DQN in keras? especially how to implement clipping the reward?

Is this pseudo code correct:

if reward<-threshold reward=-1
elseif reward>threshold reward=1
elseif -threshold<reward<threshold reward=reward/threshold


And if reward is always positive how we can change clipping the reward?

Since you're using keras-rl, you could use its class Processor. And simply write a new processor and assign to your agent. The new processor will be something like:

class MyProcessor(Processor):
def process_reward(self, reward):
"""Processes the reward as obtained from the environment for use in an agent and
returns it.

# Arguments
reward (float): A reward as obtained by the environment

# Returns
Reward obtained by the environment processed
"""
# Change min and max according to your needs. I supposed that your threshold was 1.
min = -1
max = 1
return float(np.clip(reward, min, max))


Note that there isn't a real need to cast the reward as a float and in some agent implementation it could also fail (but it's wrong the agent implementation if it does). I've done it because in Gym the reward is defined to be a float. As previously said, usually nothing goes wrong if it is an int, a numpy.float64, or something else but easily castable as a float.