# Prioritized Experience Replay - for whole episodes

I want to use Prioritized Experience Replay for whole episodes, instead for single transitions.

What's the best way to define the priorities - as episodes can be of different lengths? Personally I can think of summing up TD(0) / TD(n) errors or averaging them. What would this mean for hyperparameters like α and β?

Is there actually a 'correct way' from theoretical perspective?

• What return estimate are you using? When I saw the title I thought it would be Monte Carlo (or TD(1)) - is it single step TD (or TD(0)) or some intermediate step size (e.g. TD($\lambda$))? This could make a difference theoretically - although I am not sure whether there is a strong theoretical backing even to normal prioritized experience replay, it's more of an engineering approach backed by good empirical results. – Neil Slater Sep 19 at 7:00
• Currently I'm using TD(0), but I'm planning to extend it to n-step TD / TD(n) (but not TD(λ)). The reason why I want to store whole episodes is the recurrent nature of my data (and my model), so the model needs to replay whole episodes. – ScientiaEtVeritas Sep 19 at 9:20