Although RL algorithms can be run online, in practice this is not stable when learning off policy (as in Q-learning) and with a function approximator. To avoid this, new experience can be added to history and the agent learn from the history (called experience replay). You could think of this as a semi-online approach, since new data is immediately available to learn from, but depending on batch sizes and history size, it might not be used to alter parameters for a few time steps.
Typically in RL systems like DQN, you would train for some randomly sampled batch between each action, out of some window of historical data (maybe all historical data, maybe last N steps). The amount of training you perform between actions is a hyper-parameter of your model, as is any sampling bias towards newer data.
For example in the Atari game-playing paper by Deep Mind team, the agent sampled a mini-batch with 32 observations (state, action, reward, next state) to train the neural network on, in between each action, whilst playing the game online.
The concept of an epoch does not occur in online learning. If you are using each epoch to report performance metrics, and want to continue using comparable numbers, then you can pick some number of training iterations instead. There are no fixed rules for this, but you might want to consider reporting the same statistics on similar number of iterations as you trained on historical data - e.g. if you had 10,000 training samples in your history and are now training online with a mini-batch size of 50 per time step, then report progress every 10,000/50 = 200 time steps.
