1
$\begingroup$

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or if I'm not understanding how to read the results?

batch_size = 16
img_height = 127
img_width = 127
channel = 3 #RGB
train_dataset = image_dataset_from_directory(Train_data_dir,
                                             shuffle=True,
                                             batch_size=batch_size,
                                             image_size=(img_height, img_width),
                                             class_names = class_names)

##Transfer learning code from mobilenetV2/imagenet here to create model

initial_epochs = 10
history = model.fit(train_dataset, validation_data=test_dataset, epochs=initial_epochs)


train accuracy: 0.8030709732662548 from history.history['accuracy']
train loss: 0.26204305738210676 from history.history['loss']

test accuracy: 0.6920163252136924 from history.history['val_accuracy']
test loss: 0.6397175192832947 from history.history['val_loss']

The below are the shapes and datatypes of the datasets and how I'm passing them to the model.fit:

train_dataset: Found 5216 files belonging to 2 classes. <BatchDataset shapes: ((None, 127, 127, 3), (None,)), types: (tf.float32, tf.int32)>

test_dataset: Found 624 files belonging to 2 classes. <BatchDataset shapes: ((None, 127, 127, 3), (None,)), types: (tf.float32, tf.int32)>

initial_epochs = 10
history = model.fit(train_dataset, validation_data=test_dataset, epochs=initial_epochs)

Then I tried to check the evaluate method on the test_dataset and I'm not sure if what I'm doing is correct, but it doesn't seem to match what the fit method results give:

score = model.evaluate(test_dataset, verbose=0)

print(score) print(f'Test loss: {score[0]} / Test accuracy: {score[1]}')

Test loss: 0.6872978806495667 / Test accuracy: 0.7644230723381042

Are the results supposed to match for the fit method and the evaluate method?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

found another post asking the same question: https://stackoverflow.com/questions/44843581/what-is-the-difference-between-model-fit-an-model-evaluate-in-keras

Although I'm still not sure if the values should be the same for the validation dataset if it was passed in to the fit and evaluate methods. I've heard they may not be the same.

Improve this answer
$\endgroup$
0
$\begingroup$

If you are using tf.keras.preprocessing.image.ImageDataGenerator for loading data, try setting shuffle=False for test data.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Hi, I'm using the keras.preprocessing.image_dataset_from_directory() to load both the training dataset and the validation dataset. The images are already separated in different folders training, test, and validation. I passed the dataset labeled test_dataset as validation_data=test_dataset during the fit(). Then used the same test_dataset for the evaluate() method to see if the loss and acc would be the same. I'll try setting shuffle $\endgroup$
    – ptn77
    Commented Apr 9, 2023 at 19:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.