How to analyze neural network quality in case of overfitting?

Question

I have a Keras neural network that has images both as input and reference data.

My network demonstrates overfitting (for example, train accuracy is about 80% but test accuracy is only up to 70%) due to small amount of input data relatively to network size. Nevertheless, 70% test accuracy is pretty much fine for my problem and test accuracy doesn't start decreasing after overfitting begins -- it increases along with train accuracy, but is always several percent behind it. And model simplification doesn't allow me to reach such good accuracy.

So, in my case it looks like overfitting is doing nothing bad, but I know that its observed effects may be quite different and I'd like to check if my model performs correctly.

Could you please suggest some classical ways to analyze the model quality? May be special metrics or statistical methods are usually used or some sort of entropy parameters estimation needs to be performed? Is it possible to understand whether my model learned some useful patterns from input data or just memorized the data itself?

Answer 1

A quite useful site is this https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/. Some other things that you could do in order to avoid overfitting is:

1. early stoping, plot the accuracy of validation and test data sets over the time and when the validation accuracy becomes much better stop training your model.

2. try to use L2 regularization which gives a penalty of the complexity of the model.

3. in case that you use trees, try to prune your tree.

4. in case of neural networks try to use dropout, https://machinelearningmastery.com/dropout-regularization-deep-learning-models-keras/.