Are batch iteration and epochs different in reinforcement learning compared to supervised learning?

Question

I'm following the Udacity "AWS DeepRacer" course about self driving cars with reinforcement learning. In one lesson, they says this:

Batch size - This determines how many images, randomly sampled from the most recent episode, are used for training before updating the model.

Epochs - Determines how many times you will loop through the batched data before updating the training weights. A larger number of epochs is likely needed if your model is still improving but training has otherwise ceased.

Learning rate - Controls the speed at which your algorithm learns (it enlarges or shrinks the weight update after each epoch).

Using Tensorflow and Keras with Adam, I learned that the weights are updated after each batch iteration. But in this course, they says that the weights are updated only after each epoch, and the batch size determine how much of the training data to loop through before updating the model. So, what's the difference between updating the model and updating the weights? In tensorflow with keras, the weights are updated after each batch iteration and also after each epoch? Is it different compared to RL?

Answer 1

Your initial statements are correct.

Epoch is one-single pass over the full training set.

Most people use the word Batch as the number of samples used for one update of the weights. (The back-propagation process calculates the gradients for every single sample in the batch, but the weights update is performed a single time for the mean of the gradients over the batch).

However, originally the word batch was used to mean a training process where the batch size is equal to the overall number of samples in the training set (meaning that the weights update is performed only once per epoch). What we now refer to as "batch", was originally named "mini-batch".

Over the years, the word batch became synonymous with mini-batch (batch training is very rare these days). I think that this is the source to your confusion.