Well there are a number of ways you can improve a CNN network without changing the architecture.
So I will try to explain each of them as much as I can also since your Validation Loss is diverging from your Training Loss quiet early on which indicates Overfitting but you have figured that out already.
- How To Replicate The Result From The Paper: Use the code as is without changing anything. Also use the same dataset that they have used for training. This way you should be able to replicate their training results. Then produce the output on the same data as they have to check if you are getting similar results. Of-course keep in mind both the training and prediction results that you get and the paper have might be different but they should be within some acceptable range.
Now if you want to train the model on your own data and that is causing the issue you can try to look into the following:
- Check Your Data: Since generally the model given in the paper is suppose to work readily for some other kind of data as well without changing anything the first thing we should consider is that something might be wrong with the data that we are providing the model with. It can be a number of things 1) Very few data points for training hence the reason for Overfitting 2) Even though you have large number of data points make sure that you are providing the model different data points in each epoch and not the same one which can cause Overfitting. 3) Data is just too simple, try some data augmentation techniques to increase the variance in the training set. There can be more but I would suggest you look at these first.
If you are satisfied that nothing is wrong with the training data then lets move on the next things that we can try:
- Change Optimizer: If your model is overfitting it means either you are getting stuck in some local minima and or you have too high learning rate. But let's talk about optimizer in this step. Try changing default optimizer to one that is much less aggressive, in my knowledge the optimizers from less to most aggressive goes something like this Gradient Descent--> Stochastic Gradient Descent---> Stochastic Gradient Descent with Momentum---> Adagrad ----> RMSProp ----> Adam. So maybe you can start with this list in sequence and if something improves.
- Change Learning Rate: Since the model is overfitting we can try to reduce the learning rate and see if training loss matches with the validation loss and comes down together. General Good Values for learning rate can be 0.0001 ---> 0.003 ----> 0.001 ----> 0.003 and so on...
- Batch Size: Of Course if your batch size if too low your model will fit perfectly for the training set in that epoch but for the same epoch the validation loss will be too high. So try to keep maximum batch size that your system can handle is all I can say...
There are many more advance things that we can try but I am guessing these should fix the problem you are facing. Hope this will be of use and would love to hear your feedback if any of these things helped you out or not :)