I'm trying to classify fonts in images into 7 classes.
I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the following model:
resnet = tf.keras.applications.resnet50.ResNet50(include_top=False, weights='imagenet', input_shape=(224,224,3))
# Freeze the layers which you don't want to train. Here I am freezing the all layers.
resnet.trainable = False
global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
prediction_layer = tf.keras.layers.Dense(len(label_encoder.classes_), activation='softmax')
inputs = tf.keras.Input(shape=(224, 224, 3))
x = resnet(inputs, training=False)
x = global_average_layer(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = prediction_layer(x)
fonter = tf.keras.Model(inputs, outputs)
base_learning_rate = 0.1
fonter.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),
loss = "categorical_crossentropy",
metrics=['accuracy'])
Note: I've tried learning rates from 0.0001 to 0.1, same results.
I have a generator function that creates batches of 1 image and label for the model to fit. Each one of the images is a cropped image from a bounding box of some letter:
history = fonter.fit(train_data_and_labels,
epochs=initial_epochs,
validation_data=val_data_and_labels,
max_queue_size=1,
steps_per_epoch=data_size(db, X_train),
validation_steps=data_size(db, X_val)
)
>>>
Epoch 1/10
20358/20358 [==============================] - 608s 30ms/step - loss: 2684.4944 - accuracy: 0.1371 - val_loss: 5525.3306 - val_accuracy: 0.1093
Epoch 2/10
20358/20358 [==============================] - 588s 29ms/step - loss: 9393.9014 - accuracy: 0.1317 - val_loss: 12084.6680 - val_accuracy: 0.1076
Epoch 3/10
20358/20358 [==============================] - 588s 29ms/step - loss: 17228.0195 - accuracy: 0.1245 - val_loss: 19242.8711 - val_accuracy: 0.1381
Epoch 4/10
20358/20358 [==============================] - 587s 29ms/step - loss: 25574.2637 - accuracy: 0.1263 - val_loss: 26948.8184 - val_accuracy: 0.1191
Epoch 5/10
20358/20358 [==============================] - 588s 29ms/step - loss: 34036.8086 - accuracy: 0.1221 - val_loss: 34368.6992 - val_accuracy: 0.1448
Epoch 6/10
20358/20358 [==============================] - 591s 29ms/step - loss: 42678.4570 - accuracy: 0.1205 - val_loss: 42555.9531 - val_accuracy: 0.1315
Epoch 7/10
20358/20358 [==============================] - 591s 29ms/step - loss: 51241.7305 - accuracy: 0.1245 - val_loss: 50239.5547 - val_accuracy: 0.1306
Epoch 8/10
20358/20358 [==============================] - 590s 29ms/step - loss: 59967.1719 - accuracy: 0.1245 - val_loss: 58033.8398 - val_accuracy: 0.1217
Epoch 9/10
20358/20358 [==============================] - 577s 28ms/step - loss: 68954.7969 - accuracy: 0.1221 - val_loss: 65791.9844 - val_accuracy: 0.1284
Epoch 10/10
20358/20358 [==============================] - 572s 28ms/step - loss: 78013.2812 - accuracy: 0.1192 - val_loss: 73505.6719 - val_accuracy: 0.1191
The model's loss is increasing on both the training and the validation and the accuracy doesn't increase at all.
Does anyone have a clue why would the model perform so poorly on both the accuracy and loss function (of course related)?
steps_per_epoch
as the number of samples in the training dataset, which is incorrect. The valid expression issteps_per_epoch = num_samples / batch_size
. Select a batch size, and setsteps_per_epoch
accordingly. $\endgroup$