# Different Kernel Initializers in my prediction layer with Transfer Learning could affect performance?

So I have this model right here and the task is to classify 3 labels.:

def ResNet50(image_shape = IMG_SIZE, data_augmentation=data_augmenter()):

input_shape = image_shape + (3,)

# Remove top layer in order to put mine with the correct classification labels, get weights for imageNet
base_model = tf.keras.applications.resnet_v2.ResNet50V2(input_shape=input_shape, include_top=False, weights='imagenet')

# Freeze base model
base_model.trainable = False

# Define input layer
inputs = tf.keras.Input(shape=input_shape)

# Apply Data Augmentation
x = data_augmentation(inputs)

# Preproccess input using the same weights base model was trained on
x = tf.keras.applications.resnet_v2.preprocess_input(x)

# Set training = False to avoid keeping track of statistics in the batch norm layer
x = base_model(x, training = False)

# Add avaragePooling
x = tfl.GlobalAveragePooling2D()(x)

# Apply Dropout with 20% chance
#x = tfl.Dropout(rate = 0.2)(x)

# Add prediction/output layer with 3 neurons (Class Number = 3)
outputs = tfl.Dense(3)(x)

model = tf.keras.Model(inputs, outputs)

return model


The only thing I changed was to add Global AveragePooling before my prediction layer, and I also have a Dropout layer in case the model starts overfitting. So far it did not overfit and that's why I have it as a comment.

I did 2 runs so far and the model did not go beyond 83% accuracy with Adam and Weight Decay by a factor of 5 (0.2).

model.compile(optimizer=Adam(learning_rate = 0.001), loss=CategoricalCrossentropy(from_logits = True), metrics=["accuracy"])

reduce_LR_cb = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3)


My runs so far:

# Initial LR Weight Decay Epochs Run kernel_initializer(predition layer) Optimizer Val Accuracy Result
1 0.0001 0.2 35 Glorot Uniform(Default) Adam 0.8139 Failed
2 0.0004 0.2 32 Glorot Uniform(Default) Adam 0.8363 Failed
3 0.001 0.2 23 Glorot Uniform(Default) Adam 0.8364 Failed
4* 0.004 0.2 Glorot Uniform(Default) Adam

'*' In progress

What I am thinking is to add a kernel initializer at my prediction layer if run 3 does not work out as well. I know that the default initializer is GlorotUnirform, but is it possible for a different initializer to boost my performance? Like HeNormal, since that's the one they used in ResNet paper?