I'm doing a transfer learning with ResNet50. My dataset is clothes(224x224x3), and

49 category(classes) -> training data 1000 per 1 category, total 49000. And

valid data 200 per 1 category, total 9800.

All the data are standardized (include valid data) . So I think the data unbalance and the standardization is not a problem. Detail code is below this line.

(Ignore the variable names(ex.nasnet) please).

from keras.applications import NASNetMobile, ResNet50, VGG19, VGG16
from keras.models import Model
from keras.layers import Dense, AveragePooling2D, Dropout, Input, Flatten, BatchNormalization
from keras.optimizers import Adam, SGD

from datagenerator import DataGenerator
import numpy as np
import os

# Set Parameters and Max File Count
params = {'dim': (224,224),
          'batch_size': 64,
          'n_classes': 49,
          'n_channels': 3,
          'shuffle': True}

file_cnt = 49000
max_cnt = 58800

# Datasets

train = np.arange(file_cnt)
test = np.arange(file_cnt,max_cnt)
dataset = np.append(train,test)

# Generators
training_generator = DataGenerator(train, **params)
validation_generator = DataGenerator(test, **params)

# get resnet layers and weights, add FCN
nasnet_model = ResNet50(weights='imagenet', include_top=False,input_tensor=Input(shape=(224, 224, 3)))    

nasnet_len = len(nasnet_model.layers)

x = nasnet_model.output
x = AveragePooling2D()(x)
x = Flatten(name="flatten")(x)
x = Dense(512, activation="relu")(x)
x = BatchNormalization()(x)
x = Dropout(0.5)(x)
x = Dense(512, activation="relu")(x)
x = BatchNormalization()(x)
x = Dropout(0.5)(x)
predictions = Dense(49, activation="softmax")(x)

model = Model(inputs=nasnet_model.input, outputs=predictions)

layer_num = len(model.layers)
for layer in model.layers[0:nasnet_len]:
    layer.trainable = False

for layer in model.layers[nasnet_len:]:
    layer.trainable = True


# training
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(optimizer=Adam(lr=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])
                    validation_data=validation_generator, epochs = 20)

# saving
model_json = model.to_json()
with open("model.json", "w") as json_file : 


from keras.utils import to_categorical, Sequence

import numpy as np
import cv2
import data

class DataGenerator(Sequence): # Sequence 에 대해서 공부하
    'Generates data for Keras'
    def __init__(self, list_IDs, batch_size=64, dim=(224,224), n_channels=3,
                 n_classes=49, shuffle=True):
        self.dim = dim
        self.batch_size = batch_size
        self.list_IDs = list_IDs
        self.n_channels = n_channels
        self.n_classes = n_classes
        self.shuffle = shuffle

    def __len__(self):
        'Denotes the number of batches per epoch'
        return int(np.floor(len(self.list_IDs) / self.batch_size))

    def __getitem__(self, index):
        'Generate one batch of data'
        # Generate indexes of the batch
        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]

        # Find list of IDs
        list_IDs_temp = [self.list_IDs[k] for k in indexes]

        # Generate data
        X, y = self.__data_generation(list_IDs_temp)

        return X, y

    def on_epoch_end(self):
        'Updates indexes after each epoch'
        self.indexes = np.arange(len(self.list_IDs))
        if self.shuffle == True:

    def __data_generation(self, list_IDs_temp):
        'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)
        # Initialization
        X = np.empty((self.batch_size, *self.dim, self.n_channels))
        y = np.empty((self.batch_size), dtype=int)

        # Generate data
        for i, ID in enumerate(list_IDs_temp):
            # Store sample
            X[i,] = np.load('desktop/standardized_data/input' + str(int(ID)) + '.npy')
            # Store class
            y[i] = np.load('desktop/standardized_data/output' + str(int(ID)) + '.npy')
            #tmp = self.labels[int(ID)] 
        return X, to_categorical(y, num_classes=self.n_classes)

But the valid loss and accuracy is like this.

Epoch 6/20
765/765 [==============================] - 812s 1s/step - loss: 0.8220 - accuracy: 0.7020 - val_loss: 6.2641 - val_accuracy: 0.0345
Epoch 7/20
765/765 [==============================] - 836s 1s/step - loss: 0.7322 - accuracy: 0.7254 - val_loss: 6.8545 - val_accuracy: 0.0394
Epoch 8/20
765/765 [==============================] - 822s 1s/step - loss: 0.6664 - accuracy: 0.7432 - val_loss: 6.6525 - val_accuracy: 0.0362
Epoch 9/20
765/765 [==============================] - 799s 1s/step - loss: 0.6098 - accuracy: 0.7597 - val_loss: 6.1669 - val_accuracy: 0.0346
Epoch 10/20
764/765 [============================>.] - ETA: 0s - loss: 0.5719 - accuracy: 0.7711  

Why is this happening? ( epoch 1~5 was missing, but the valid loss and accuracy were same, and the training loss and accuracy was getting better every time. )

Now I was trying to test with some training data, but the result was the same. That is, I trained with training data, and tested with training data, but accuracy difference was same.


1 Answer 1


As you can see from the Train and Validation loss (and also accuracy). While your model is able to learn, your validation results do not improve. This means underfitting, or in this specific case, your validation data is different from your train data.

The reason is following:

training_generator = DataGenerator(train, **params)
validation_generator = DataGenerator(test, **params)

DataGenerator() creates random data (as I saw in its documentation). Since you generated your data using it twice, it created two different random data. Since your params are the same in both, it did not give an error, thus you did not notice. Thus, your validation result is low as expected.

Just do something like this:

generatedData = DataGenerator(dataRange, **params)

Then, split your generatedData to train and validation (also test maybe).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.