0
$\begingroup$

I am creating a custom gym environment that should abstract a non-episodic/continuous task.

Generally gym requires to return obs, reward, done, info after each step:

obs, reward, done, info = env.step(actions)

Now, what if there will never be a termination signal (done)? Should my implementation always return done = False or should it always return done = True? Would returning one or the other make a difference for the RL-algorithms that are using it (particularly looking at stable-baselines here)?

$\endgroup$

1 Answer 1

1
$\begingroup$

There is a big difference. If your done flag was always True, then you would always have episodes of length one, which is the opposite of what you want.

What the updated version of Gym, Gymnasium, has introduced, is a terminated and truncated flag. Rather than receive a single done flag we are now told whether the termination was a true termination, i.e. we have reached a terminal state, or whether the episode is being truncated. This is useful for continuing tasks like you mentioned because you can reset the environment, which is typically good for exploration, but still bootstrap from your next state.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.