# How to transition between offline and online learning?

I am training an RL agent on a time series (with TensorFlow in python) in the following way: to predict the quantity of interest at time period $t$, I feed a window of $W$ observations at time periods $[t-W,t)$. Throughout the training the window advances step by step until I have a minibatch of $M$ observations and rewards to train on. Repeat until you run out of historical data, that's one Epoch. I train on a few thousand epochs with small learning rate (the loss is very unstable).

Eventually, I want to start pulling live data from the environment to make predictions. At this point, if I wanted to continue the training online, how should I deal with the epochs? There is no "running out of data" anymore.