I would like to use a custom generator so that I can implement custom augmentations on my dataset in Keras. However, I built a generic generator (without augmentation) and am confused why it is performing way worse than the built-in fit function.
My generator looks like this:
def data_generator_plain(features, labels, batch_size):
batch_features = np.zeros((batch_size, IMG_SIZE, IMG_SIZE, NUM_CHANNELS), dtype=np.float64)
batch_labels = np.zeros((batch_size, NUM_KEYPOINTS * 2), dtype=np.float64)
while True:
steps = len(batch_features) // batch_size
for i in range(steps):
for j in range(batch_size):
batch_features[j] = features[(i*batch_size)+j]
batch_labels[j] = labels[(i*batch_size)+j]
yield batch_features, batch_labels
and when i call model.fit_generator(data_generator_plain(X_train, y_train, BATCH_SIZE), steps_per_epoch=X_train.shape[0]//BATCH_SIZE, epochs=NUM_EPOCHS, validation_data=(X_val, y_val))
the val_accuracy ends around 73% while the basic model.fit(X_train, y_train, epochs=NUM_EPOCHS, batch_size=BATCH_SIZE, validation_data=(X_val, y_val))
val_accuracy ends around 93%.
Is my generator implementation correct? In theory, it should perform the same as the built-in fit function, right? I imagine it is not showing the algorithm all the examples it needs to. Thank you!
shuffle=True
by default. If X_train.shape[0] is not a multiple of BATCH_SIZE, it would be another small discrepancy. Or maybe there is a bug in some other part of your code. You can always insert some print statements and manually debug such problems. $\endgroup$