Skip to main content
1 of 3
Charles
  • 191
  • 2
  • 7

Keras intuition/guidelines for setting epochs and batches

I'm using Python with Keras to make a convolutional neural network (CNN) for an image classifier. I took about 50 images of documents and 150 images of non-documents for training. I shrunk the resolution, using scipy.misc.imresize(small, (32,32)) to standardize the images, when looking through these pixelated images I thought that I could still tell the difference between photos and documents, so I figured that the ML algorithm should be able to as well.

My question is in regards to the number of epochs and batches. I've seen this question regarding the topic here:

How to set batch_size, steps_per epoch, and validation steps?

But the answer was very definition-based; I'm looking for intuition. I would like to know if there are general guidelines as to what values to set the number of epochs and batches to for a given problem. I performed a crude parameter sweep across the number of epochs and the number of batches. Here is the CNN model:

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), input_shape=input_shape, activation='sigmoid'))
model.add(Conv2D(32, kernel_size=(3, 3), input_shape=input_shape, activation='sigmoid'))
model.add(Flatten())
model.add(Dense(num_classes, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

And here are the results:

enter image description here

Here are some observations that I've made so far. The circled region seems to be pretty good for training since high accuracy is achieved relatively early on and it does not seem to oscillate much as further epochs pass. It seems that N_batches = 1 is generally a bad idea since training does not seem to improve the model. Additionally, it seems that N_batches >~ N_epochs seems to be desirable.

Question 1: is N_batches = 1 generally a bad? Or is this just for my particular parameter sweep?

Question 2: Is it generally the case that N_epochs >~ N_batches is recommended?

Question 3: Can anyone speak to the robustness of the generalizations in questions 2?

Thank you.

Charles
  • 191
  • 2
  • 7