Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have a very basic question on the optimization algotithm, when I'm adjusting weights and biases in a NN, should I:
iterations_number times.or
iterations_number times and then continue with the next batch.One iteration means that you do one forward pass and one backward pass for a single batch containing batch_size examples. Then you move on to the next batch. Also see this post on SE SO.
Note that this is not identical to an epoch. An epoch is only completed when all examples of your dataset have been passed through your network. And as long as your batch size is less than the number of datapoints you will need $\frac{n}{\text{batch size}}$ (with $n$ being the sample size) iterations to complete an epoch. Also see this answer on SE DS.
epochs_number instead of iterations_number as the second is just $\frac{n}{batch size}$.
$\endgroup$
Jan 16, 2021 at 18:13
iterations_number = (n / batchsize) * epochs_number.
$\endgroup$