0
$\begingroup$

I am training a deep learning network using MATLAB and would like to increase the number of iterations per epoch.

Using trainingOptions

options = trainingOptions('sgdm', ...
    'Momentum',0.9,...
    'MaxEpochs',100, ...
    'ValidationData',{XValidation,YValidation}, ...
    'MiniBatchSize', 64,...
    'Verbose',false, ...
    'Plots','training-progress');

I get the following, where the number of iterations per epoch is set automatically to 1. Therefore 1 pass of 64 training examples is needed for one epoch. How can I increase this to, for example 10 iterations per epoch?

enter image description here

According the the docs:

An iteration corresponds to a mini-batch.

As I understand, the number of iterations is the number of passes, each pass using 64 (batch size) number of examples. So in the above, I have 1 pass of 64 training examples for 100 epochs.

Does this mean the iterations per epoch are set automatically according to how many training examples there are present? Can I alter this parameter?

$\endgroup$
1
$\begingroup$

I think you misunderstand the difference between the terms:

Epoch - A single pass over the entire training dataset.

Mini-batch - Number of samples to pass thru the model before performing a single update of its weights.

Iterations - Number of required mini-batches to complete a full epoch.

Example: you have a training dataset with 256 samples, a mini-batch size of 32 and epoch number 100. Each training sample will be viewed by the model 100 times (as per the epoch number). Each epoch will be done in 8 iterations (i.e. the model will be updated 8 times in an epoch). Each iteration will include 32 training samples passed thru the model. The back propagation algorithm will calculate the required change in the model weights for every single sample, but the actual update will take place on the mean (or sum) of all 32 samples in the mini-batch.

As you said at the end, "iterations per epoch are set automatically according to how many training examples there are present". There is no reason to alter this parameter, if you want to change the number of passes over the data, use the epoch number.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for your answer... and what happens if I set the mini batch to a number greater than the training dataset? $\endgroup$ – rrz0 Dec 3 '18 at 18:05
  • $\begingroup$ In general terms, that would be wrong. I don't know how it is implemented in the matlab function, but my guess is that they probably check for it and then simply set the mini-batch size to be equal to the size of the training dataset. $\endgroup$ – Mark.F Dec 3 '18 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.