0
$\begingroup$

I'm trying to classify fonts in images into 7 classes.

I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the following model:

resnet = tf.keras.applications.resnet50.ResNet50(include_top=False, weights='imagenet', input_shape=(224,224,3))
# Freeze the layers which you don't want to train. Here I am freezing the all layers.
resnet.trainable = False
global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
prediction_layer = tf.keras.layers.Dense(len(label_encoder.classes_), activation='softmax')
inputs = tf.keras.Input(shape=(224, 224, 3))
x = resnet(inputs, training=False)
x = global_average_layer(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = prediction_layer(x)
fonter = tf.keras.Model(inputs, outputs)

base_learning_rate = 0.1
fonter.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learning_rate),
              loss = "categorical_crossentropy",
              metrics=['accuracy'])

Note: I've tried learning rates from 0.0001 to 0.1, same results.

I have a generator function that creates batches of 1 image and label for the model to fit. Each one of the images is a cropped image from a bounding box of some letter:

history = fonter.fit(train_data_and_labels,
                    epochs=initial_epochs,
                    validation_data=val_data_and_labels,
                    max_queue_size=1,
                    steps_per_epoch=data_size(db, X_train),
                    validation_steps=data_size(db, X_val)
                   )
>>>
Epoch 1/10
20358/20358 [==============================] - 608s 30ms/step - loss: 2684.4944 - accuracy: 0.1371 - val_loss: 5525.3306 - val_accuracy: 0.1093
Epoch 2/10
20358/20358 [==============================] - 588s 29ms/step - loss: 9393.9014 - accuracy: 0.1317 - val_loss: 12084.6680 - val_accuracy: 0.1076
Epoch 3/10
20358/20358 [==============================] - 588s 29ms/step - loss: 17228.0195 - accuracy: 0.1245 - val_loss: 19242.8711 - val_accuracy: 0.1381
Epoch 4/10
20358/20358 [==============================] - 587s 29ms/step - loss: 25574.2637 - accuracy: 0.1263 - val_loss: 26948.8184 - val_accuracy: 0.1191
Epoch 5/10
20358/20358 [==============================] - 588s 29ms/step - loss: 34036.8086 - accuracy: 0.1221 - val_loss: 34368.6992 - val_accuracy: 0.1448
Epoch 6/10
20358/20358 [==============================] - 591s 29ms/step - loss: 42678.4570 - accuracy: 0.1205 - val_loss: 42555.9531 - val_accuracy: 0.1315
Epoch 7/10
20358/20358 [==============================] - 591s 29ms/step - loss: 51241.7305 - accuracy: 0.1245 - val_loss: 50239.5547 - val_accuracy: 0.1306
Epoch 8/10
20358/20358 [==============================] - 590s 29ms/step - loss: 59967.1719 - accuracy: 0.1245 - val_loss: 58033.8398 - val_accuracy: 0.1217
Epoch 9/10
20358/20358 [==============================] - 577s 28ms/step - loss: 68954.7969 - accuracy: 0.1221 - val_loss: 65791.9844 - val_accuracy: 0.1284
Epoch 10/10
20358/20358 [==============================] - 572s 28ms/step - loss: 78013.2812 - accuracy: 0.1192 - val_loss: 73505.6719 - val_accuracy: 0.1191

The model's loss is increasing on both the training and the validation and the accuracy doesn't increase at all.

Does anyone have a clue why would the model perform so poorly on both the accuracy and loss function (of course related)?

Improve this question
$\endgroup$
2
  • $\begingroup$ You're setting steps_per_epoch as the number of samples in the training dataset, which is incorrect. The valid expression is steps_per_epoch = num_samples / batch_size. Select a batch size, and set steps_per_epoch accordingly. $\endgroup$
    – Shubham Panchal
    Jan 9, 2022 at 1:35
  • $\begingroup$ @ShubhamPanchal My batch size is 1, so the steps are actually the number of samples (generator functions return batches). Is there a reason it would affect the training? $\endgroup$
    – Ofek .T.
    Jan 9, 2022 at 8:05

1 Answer 1

Reset to default
0
$\begingroup$

The problem was I didn't one-hot encode my labels, so my model did not train correctly.

I.E I've given my model the label [4] instead of [0,0,0,0,1,0,0] (because there are 7 classes and the label is the fifth)

I found out when I switched from using a generator to numpy array, then tensorflow prompt me with an error.
I don't understand why didn't tensorflow prompt me with an error while using a generator..

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.