Why the validation accuracy does not increase in a normal way over the epochs?

I'm trying to transfer learning VGG16 model with imagenet in a dataset of retinal images but i'm confused to get a graph like this in the picture below, I don't know why the validation accuracy didn't increase in a normal way over the epochs, like training accuracy did, is it an index of overfitting ? if yes, how can i overcome it ?

STEP_SIZE_TRAIN = len(train_generator)
STEP_SIZE_TEST = len(test_generator)

Network = ModelDefinition(base_model)

# don't train existing weights
for layer in Network.layers:
layer.trainable = False
# Number of classes
folders = glob(train_path + '/*')
# our layers
x = Flatten()(Network.output)
# a layer instance is callable on a tensor, and returns a tensor
prediction = Dense(len(folders), activation='softmax')(x)

# This creates a model that includes
# the Input layer and One Dense layers
model = Model(inputs=Network.input, outputs=prediction)
#view the structure of the model
model.summary()
#tell the model what cost and optimization method to use
model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

r = model.fit_generator(train_generator, steps_per_epoch=STEP_SIZE_TRAIN, validation_data=test_generator,
validation_steps=STEP_SIZE_TEST, epochs=40)

Model_fileName = 'KERAS/MODELS/all-D0-D14/' + base_model
train_datagen = ImageDataGenerator(rescale=1. / 255, shear_range=0.2,
zoom_range=0.2, horizontal_flip=True)
train_generator = train_datagen.flow_from_directory(train_path, target_size=
(224, 224), batch_size=32,class_mode='categorical')

test_datagen = ImageDataGenerator(rescale=1. / 255)
val_generator = test_datagen.flow_from_directory(test_path, target_size=(224,
224), batch_size=32,class_mode='categorical', shuffle=True, seed=42)

• What code are you using to train your VGG model? – Oxbowerce Apr 13 at 11:14
• Could you share the code for this chart, looks very weird. Mostly model after certain epochs stopped learning and the line becomes steady at some value – 10xAI Apr 13 at 12:11
• I edited the post, please see the code above. – ELbafa Apr 13 at 14:37