0
$\begingroup$

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size 5000.

My implementation of boltzmann exploration:

def get_action(state,episode,temperature = 1):
    state_encod = np.reshape(state, [1, state_size])
    q_values = model.predict(state_encod)         
    prob_act = np.empty(len(q_values[0]))

    for i in range(len(prob_act)):
        prob_act[i] = np.exp(q_values[0][i]/temperature)

    #numpy matrix element-wise division for denominator (sum of numerators)
    prob_act = np.true_divide(prob_act,sum(prob_act))

    action_q_value = np.random.choice(q_values[0],p=prob_act)
    action_keys = np.where(q_values[0] == action_q_value)
    action_key = action_keys[0][0]
    action = index_to_action_mapping[action_key]
    return action    

If my temperature variable is 200, after 100 episodes I get an error

ValueError: probabilities contain NaN

If my temperature is 1 in very few episodes i get NaN error.

Why is this happening. Am I doing something wrong here? How to select the temperature variable? Can someone help me with this.

Thanks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.