43
$\begingroup$

I am starting to learn CNNs using Keras. I am using the theano backend.

I don't understand how to set values to:

  • batch_size,
  • steps per epoch,
  • validation_steps.

What should be the value set to batch_size, steps per epoch, and validation steps if I have 240,000 samples in the training set and 80,000 in the test set?

$\endgroup$
  • $\begingroup$ What's your hardware specifications? It depends on that Generally people use batch size of 32/64 , epochs as 10~15 and then you can calculate steps per epoch from the above.. $\endgroup$ – Aditya Mar 30 '18 at 9:49
44
$\begingroup$
  • batch_size determines the number of samples in each mini batch. Its maximum is the number of all samples, which makes gradient descent accurate, the loss will decrease towards the minimum if the learning rate is small enough, but iterations are slower. Its minimum is 1, resulting in stochastic gradient descent: Fast but the direction of the gradient step is based only on one example, the loss may jump around. batch_size allows to adjust between the two extremes: accurate gradient direction and fast iteration. Also, the maximum value for batch_size may be limited if your model + data set does not fit into the available (GPU) memory.
  • steps_per_epoch the number of batch iterations before a training epoch is considered finished. If you have a training set of fixed size you can ignore it but it may be useful if you have a huge data set or if you are generating random data augmentations on the fly, i.e. if your training set has a (generated) infinite size. If you have the time to go through your whole training data set I recommend to skip this parameter.
  • validation_steps similar to steps_per_epoch but on the validation data set instead on the training data. If you have the time to go through your whole validation data set I recommend to skip this parameter.
| improve this answer | |
$\endgroup$
  • $\begingroup$ What do you mean by "skipping this parameter"? When I remove the parameter I get When using data tensors as input to a model, you should specify the steps_per_epoch argument. $\endgroup$ – Nicolas Raoul Sep 27 '18 at 7:09
  • $\begingroup$ According to the documentation, the parameter steps_per_epoch of the method fit has a default and thus should be optional: "the default None is equal to the number of samples in your dataset divided by the batch size, or 1 if that cannot be determined." Source: keras.io/models/model $\endgroup$ – Silpion Sep 28 '18 at 21:04
5
$\begingroup$

there is an answer in Github

  1. model.fit_generator requires the input dataset generator to run infinitely.
  2. steps_per_epoch is used to generate the entire dataset once by calling the generator steps_per_epoch times
  3. whereas epochs give the number of times the model is trained over the entire dataset.
| improve this answer | |
$\endgroup$
0
$\begingroup$

From tensorflow_estimator/python/estimator/training.py

Stop condition:

In order to support both distributed and non-distributed configuration reliably, the only supported stop condition for model training is train_spec.max_steps. If train_spec.max_steps is None, the model is trained forever. Use with care if model stop condition is different. For example, assume that the model is expected to be trained with one epoch of training data, and the training input_fn is configured to throw OutOfRangeError after going through one epoch, which stops the Estimator.train. For a three-training-worker distributed configuration, each training worker is likely to go through the whole epoch independently. So, the model will be trained with three epochs of training data instead of one epoch.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.