0
$\begingroup$

So I have this model right here and the task is to classify 3 labels.:

def ResNet50(image_shape = IMG_SIZE, data_augmentation=data_augmenter()):

    input_shape = image_shape + (3,)

    # Remove top layer in order to put mine with the correct classification labels, get weights for imageNet
    base_model = tf.keras.applications.resnet_v2.ResNet50V2(input_shape=input_shape, include_top=False, weights='imagenet')

    # Freeze base model
    base_model.trainable = False

    # Define input layer
    inputs = tf.keras.Input(shape=input_shape)

    # Apply Data Augmentation
    x = data_augmentation(inputs)

    # Preproccess input using the same weights base model was trained on
    x = tf.keras.applications.resnet_v2.preprocess_input(x)

    # Set training = False to avoid keeping track of statistics in the batch norm layer
    x = base_model(x, training = False)
    
    # Add avaragePooling
    x = tfl.GlobalAveragePooling2D()(x)

    # Apply Dropout with 20% chance
    #x = tfl.Dropout(rate = 0.2)(x)

    # Add prediction/output layer with 3 neurons (Class Number = 3)
    outputs = tfl.Dense(3)(x)

    model = tf.keras.Model(inputs, outputs)

    return model

The only thing I changed was to add Global AveragePooling before my prediction layer, and I also have a Dropout layer in case the model starts overfitting. So far it did not overfit and that's why I have it as a comment.

I did 2 runs so far and the model did not go beyond 83% accuracy with Adam and Weight Decay by a factor of 5 (0.2).

model.compile(optimizer=Adam(learning_rate = 0.001), loss=CategoricalCrossentropy(from_logits = True), metrics=["accuracy"])

reduce_LR_cb = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3)

My runs so far:

# Initial LR Weight Decay Epochs Run kernel_initializer(predition layer) Optimizer Val Accuracy Result
1 0.0001 0.2 35 Glorot Uniform(Default) Adam 0.8139 Failed
2 0.0004 0.2 32 Glorot Uniform(Default) Adam 0.8363 Failed
3 0.001 0.2 23 Glorot Uniform(Default) Adam 0.8364 Failed
4* 0.004 0.2 Glorot Uniform(Default) Adam

'*' In progress

What I am thinking is to add a kernel initializer at my prediction layer if run 3 does not work out as well. I know that the default initializer is GlorotUnirform, but is it possible for a different initializer to boost my performance? Like HeNormal, since that's the one they used in ResNet paper?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.