How to set batch_size, steps_per epoch, and validation steps?

Question

I am starting to learn CNNs using Keras. I am using the theano backend.

I don't understand how to set values to:

• batch_size
• steps_per_epoch
• validation_steps

What should be the value set to batch_size, steps_per_epoch, and validation_steps, if I have 240,000 samples in the training set and 80,000 in the test set?

Answer 1

• batch_size determines the number of samples in each mini batch. Its maximum is the number of all samples, which makes gradient descent accurate, the loss will decrease towards the minimum if the learning rate is small enough, but iterations are slower. Its minimum is 1, resulting in stochastic gradient descent: Fast but the direction of the gradient step is based only on one example, the loss may jump around. batch_size allows to adjust between the two extremes: accurate gradient direction and fast iteration. Also, the maximum value for batch_size may be limited if your model + data set does not fit into the available (GPU) memory.
• steps_per_epoch the number of batch iterations before a training epoch is considered finished. If you have a training set of fixed size you can ignore it but it may be useful if you have a huge data set or if you are generating random data augmentations on the fly, i.e. if your training set has a (generated) infinite size. If you have the time to go through your whole training data set I recommend to skip this parameter.
• validation_steps similar to steps_per_epoch but on the validation data set instead on the training data. If you have the time to go through your whole validation data set I recommend to skip this parameter.

Answer 2

there is an answer in Github

1. model.fit_generator requires the input dataset generator to run infinitely.
2. steps_per_epoch is used to generate the entire dataset once by calling the generator steps_per_epoch times
3. whereas epochs give the number of times the model is trained over the entire dataset.

Answer 3

From tensorflow_estimator/python/estimator/training.py

Stop condition:

In order to support both distributed and non-distributed configuration reliably, the only supported stop condition for model training is train_spec.max_steps. If train_spec.max_steps is None, the model is trained forever. Use with care if model stop condition is different. For example, assume that the model is expected to be trained with one epoch of training data, and the training input_fn is configured to throw OutOfRangeError after going through one epoch, which stops the Estimator.train. For a three-training-worker distributed configuration, each training worker is likely to go through the whole epoch independently. So, the model will be trained with three epochs of training data instead of one epoch.