I created a two layered fully connected neural network as a part of a recommendation engine (after I use embedding layers for products and users). I have been trying to tune the hyper-parameters for the past couple of weeks now. Additionally I am using 5 fold cross validation and 'mse' as the error metric. Below is the curve for training and cross-validation error that I obtain post training on a GPU for 10 epochs.
I am not sure what really to make of this, fundamentally. The model doesn't overfit or underfit the data. What is even more perplexing is that the validation error is across five folds. Is this an inherent bias in the data and alludes to the fact that the data is structured such that a weak model does well on unseen data ?
Any pointers or comments will be greatly helpful. Thanks !