So I have this model right here and the task is to classify 3 labels.:

def ResNet50(image_shape = IMG_SIZE, data_augmentation=data_augmenter()):

    input_shape = image_shape + (3,)

    # Remove top layer in order to put mine with the correct classification labels, get weights for imageNet
    base_model = tf.keras.applications.resnet_v2.ResNet50V2(input_shape=input_shape, include_top=False, weights='imagenet')

    # Freeze base model
    base_model.trainable = False

    # Define input layer
    inputs = tf.keras.Input(shape=input_shape)

    # Apply Data Augmentation
    x = data_augmentation(inputs)

    # Preproccess input using the same weights base model was trained on
    x = tf.keras.applications.resnet_v2.preprocess_input(x)

    # Set training = False to avoid keeping track of statistics in the batch norm layer
    x = base_model(x, training = False)
    # Add avaragePooling
    x = tfl.GlobalAveragePooling2D()(x)

    # Apply Dropout with 20% chance
    #x = tfl.Dropout(rate = 0.2)(x)

    # Add prediction/output layer with 3 neurons (Class Number = 3)
    outputs = tfl.Dense(3)(x)

    model = tf.keras.Model(inputs, outputs)

    return model

The only thing I changed was to add Global AveragePooling before my prediction layer, and I also have a Dropout layer in case the model starts overfitting. So far it did not overfit and that's why I have it as a comment.

I did 2 runs so far and the model did not go beyond 83% accuracy with Adam and Weight Decay by a factor of 5 (0.2).

model.compile(optimizer=Adam(learning_rate = 0.001), loss=CategoricalCrossentropy(from_logits = True), metrics=["accuracy"])

reduce_LR_cb = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3)

My runs so far:

# Initial LR Weight Decay Epochs Run kernel_initializer(predition layer) Optimizer Val Accuracy Result
1 0.0001 0.2 35 Glorot Uniform(Default) Adam 0.8139 Failed
2 0.0004 0.2 32 Glorot Uniform(Default) Adam 0.8363 Failed
3 0.001 0.2 23 Glorot Uniform(Default) Adam 0.8364 Failed
4* 0.004 0.2 Glorot Uniform(Default) Adam

'*' In progress

What I am thinking is to add a kernel initializer at my prediction layer if run 3 does not work out as well. I know that the default initializer is GlorotUnirform, but is it possible for a different initializer to boost my performance? Like HeNormal, since that's the one they used in ResNet paper?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.