0
$\begingroup$

I'm currently learning to build a DNN binary classification model on some dataset. But when I analysed the results, I found it counter-intuitive that the training accuracy changed almost at the same rate as the validation accuracy. So did the losses. And they all reached a plateau at some point. How could I possibly improve the model performance at this stage? Here's a snippet of the results:

fold: 0 epoch: 0 batch: 0
training loss: 0.674389 validation loss: 0.67371
training accuracy: 0.656331 validation accuracy: 0.656968
Fold: 0 epoch: 0 batch: 500
training loss: 0.527997 validation loss: 0.527813
training accuracy: 0.734021 validation accuracy: 0.733639
Fold: 0 epoch: 0 batch: 1000
training loss: 0.526019 validation loss: 0.526277
training accuracy: 0.736067 validation accuracy: 0.735111
Fold: 0 epoch: 0 batch: 1500
training loss: 0.525466 validation loss: 0.525721
training accuracy: 0.736672 validation accuracy: 0.73532
Fold: 1 epoch: 0 batch: 0
training loss: 0.525416 validation loss: 0.525261
training accuracy: 0.735794 validation accuracy: 0.736488
Fold: 1 epoch: 0 batch: 500
training loss: 0.525751 validation loss: 0.525623
training accuracy: 0.735472 validation accuracy: 0.735887
Fold: 1 epoch: 0 batch: 1000
training loss: 0.525579 validation loss: 0.525459
training accuracy: 0.735538 validation accuracy: 0.736077 
$\endgroup$
  • $\begingroup$ You need to add more information like the nature of the data, model and the cost function. From a training perspective one expects to minimize loss and improve accuracy (or precision and recall) when training a model. The fact that it reach a plateau is also expected. $\endgroup$ – Grasshopper Jun 5 '17 at 12:30
  • 1
    $\begingroup$ Could you show how you split train and validation set, plus how the training method (and metrics reporting) work in your code? Are there any duplicated entries in your training data? Although it is desirable to see train vs validation accuracy rise together, it is unusual to have training and validation loss and metrics agreeing to 3 decimal places throughout training. It implies some unwanted link between training and validation logic and/or data. $\endgroup$ – Neil Slater Jun 5 '17 at 15:08
1
$\begingroup$

Generally, the fact that your training and validation performance are improving at the same rate is a good thing- this (usually) means that the algorithm is learning generalizable features of your problem space rather than overfitting to the noise of your training set.

Reaching a plateau in performance is also to be expected- it's very rare that a real-world machine learning problem can be perfectly solved, and a perfectly solved problem would be the only type that didn't reach a plateau in testing and validation performance (before it eventually did reach a plateau at 100% accuracy).

Think of the plateau as the maximum performance that can be achieved given the particular parameter values, features, and architecture of the solution. In order to achieve performance beyond your plateau values, one of these considerations will need to be adjusted.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.