Image recognition model with CNN for face gestures is really bad

Question

I have a dataset that contains facial expressions and their label, and I am trying to make a classification model for it. Unfortunatly, I can't manage to create a good model with CNN, as the highest accuracy I get is 63%. Above it, the model starts to overfit. It would be awesome if you guys could help me here. Here is some information about the data:

• 22k images in the train set, 3.5k images in the validation set, 2k images in the test set. The classes are shuffled in the dataset.
• The images are 80x80 pixels of gray scale value (made by averaging the RGB colors of the original image).
• there are 7 labels (Anger, Disgust, Fear, Happy, Neutral, Sad and Surprise)

For example, here is a (not grayscaled) image with the label "disgust":

To create the model I used CNN. I played with it a bit and the best model I got was :

model= tf.keras.Sequential()

model.add(layers.Conv2D(64, kernel_size = (3, 3), activation = 'relu', input_shape=(80, 80, 1)))
model.add(layers.MaxPool2D(pool_size = (3, 3), strides = (2, 2)))
model.add(layers.Conv2D(64, kernel_size = (3, 3), activation = "relu"))
model.add(layers.MaxPool2D(pool_size = (3, 3), strides = (2, 2)))
model.add(layers.Dropout(0.2))
model.add(layers.Flatten())
model.add(layers.Dense(128, activation = "relu"))
model.add(layers.Dense(num_classes+1, activation = "softmax"))

optimizer = tf.keras.optimizers.Adam(0.001)
model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])

Here is the output of the training before it started to overfit:

Epoch 1/10
696/696 [==============================] - 166s 238ms/step - loss: 1.2314 - accuracy: 0.5032 - val_loss: 1.0886 - val_accuracy: 0.5516
Epoch 2/10
696/696 [==============================] - 478s 687ms/step - loss: 0.9796 - accuracy: 0.6103 - val_loss: 0.9694 - val_accuracy: 0.6096
Epoch 3/10
696/696 [==============================] - 474s 680ms/step - loss: 0.8602 - accuracy: 0.6646 - val_loss: 0.9539 - val_accuracy: 0.6138
Epoch 4/10
696/696 [==============================] - 460s 661ms/step - loss: 0.7562 - accuracy: 0.7060 - val_loss: 0.9558 - val_accuracy: 0.6265
Epoch 5/10
696/696 [==============================] - 442s 635ms/step - loss: 0.6330 - accuracy: 0.7595 - val_loss: 1.0216 - val_accuracy: 0.6293

Which gave me 63% accuracy. I am trying to guess a new model, as the internet said, but I don't get any closer. I changed only the architecture; I always had the same loss function and optimizer. I tried to add image augmentation with flipping and rotations but that didn't raise the accuracy. In contrast, it actually reduced it (which is weird, so I am also suspecting that I did not implement it correctly. Though I just followed the documentation, so I assume it was good). It would be awesome to have tips from you people. If you think I should learn some theme / read a book to be able to make a good model (with an accuracy > 90%) and to develop good intuition for ML, I would be happy to do it (I read several ML books already, so I mainly need "intuition guide" now). I thank you in advance.