I am trying to understand my model by diagnosing the learning curve and how I can improve it. I aim to implement a deep-learning architecture (ResNet50) using a small dataset for binary classification problems. My dataset only consists of 896 images, which I split into training, validation, and test datasets following the 7: 1.5: 1.5 ratio.
The following image indication model is without data augmentation, but a considerable gap between the validation and training loss curve is observable.
I tried small data augmentation from what I received, the lowest validation loss, but unfortunately, it still suffers from overfitting (picture below).
Then I used more data augmentation to overcome the overfitting problem. However, it's observable from the model that the training curve is still decreasing while the validation curve seems like reached its highest capacity.
I used early stopping, which prevented the model from running further.
Transfer learning is a good solution for small datasets, and I got a pretty good result, but I wanted to solve it without using it.
So, I would like to understand more about my curve and how to improve it.
I appreciate any help you can provide.