I'm using Python with Keras to make a convolutional neural network (CNN) for an image classifier. I took about 50 images of documents and 150 images of non-documents for training. I shrunk the resolution, using
scipy.misc.imresize(small, (32,32)) to standardize the images, when looking through these pixelated images I thought that I could still tell the difference between photos and documents, so I figured that the ML algorithm should be able to as well.
My question is in regards to the number of epochs and batch size. I'm currently using
model.fit(X_train_temp, y_train_temp, epochs=N_epochs, batch_size=batch_size, verbose=verbose) . I've seen this question regarding the topic here:
But the answer was very definition-based; I'm looking for intuition. I would like to know if there are general guidelines as to what values to set the number of epochs and batch size to for a given problem. I performed a crude parameter sweep across the number of epochs and batch size. Here is the CNN model:
model = Sequential() model.add(Conv2D(32, kernel_size=(3, 3), input_shape=input_shape, activation='sigmoid')) model.add(Conv2D(32, kernel_size=(3, 3), input_shape=input_shape, activation='sigmoid')) model.add(Flatten()) model.add(Dense(num_classes, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
And here are the results:
Here are some observations that I've made so far. The circled region seems to be pretty good for training since high accuracy is achieved relatively early on and it does not seem to oscillate much as further epochs pass. It seems that
batch_size = 1 is generally a bad idea since training does not seem to improve the model. Additionally, it seems that
batch_size >~ N_epochs seems to be desirable.
Question 1: is
batch_size = 1 generally a bad? Or is this just for my particular parameter sweep?
Question 2: Is it generally the case that
N_epochs >~ batch_size is recommended?
Question 3: Can anyone speak to the robustness of the generalizations in questions 2?