2
$\begingroup$

My model's structure is

                                  Output
                                    ^
                                    |
                             ----------------
                             | Dense Network |
                             ----------------
                                   /\
                                   ||
                                   ||
                                   ||
   |--------------------|          ||          | ----------------------|  
   | RNN on features    | ========>||<======== |  Dense Network on non |
   | changing with time |        [concat]      |  time series data     |
   |--------------------|                      |-----------------------|

These are the Training and Validation set metric outputs of my model. Why are the values fluctuating so much for the validation set?. Any ideas?

[![4 Graphs. Loss, Accuracy, Precision, Recall][1]][1]

Update :

As suggested in comments I have tried increasing the validation set size Now the size ratio is 49.6%-50.4%

Also I have made the model very simple by Using fewer layers. The new graph obtained looks like this [![4 Graphs. Loss, Accuracy, Precision, Recall on simpler model][2]][2]

Is this acceptable as 'okay-fluctuating'?

$\endgroup$
5
  • $\begingroup$ Probably the test set is very small. $\endgroup$
    – 10xAI
    Jun 30 '20 at 15:37
  • $\begingroup$ @10xAI Increased the validation set size to same of train size . Still seeing almost the same behavior $\endgroup$
    – skrrrt
    Jun 30 '20 at 16:45
  • $\begingroup$ This isn't part of your question, but it maybe might give some insight into the answer. It is worth looking at your top right plot, which shows an increasing validation loss with a decreasing training loss over epochs. This looks like overfitting is happening as well. This could suggest that your proposed model is "too complex" and need regularising in some way. Maybe this could partially help in understanding the fluctuations in validation stats over epochs. $\endgroup$
    – shepan6
    Jul 3 '20 at 9:38
  • $\begingroup$ @shepan6 Okay, I will try including dropouts and make the model less complex $\endgroup$
    – skrrrt
    Jul 3 '20 at 11:24
  • $\begingroup$ @shepan6 Please check the update $\endgroup$
    – skrrrt
    Jul 3 '20 at 16:51
1
$\begingroup$

Thanks for updating the post, this level of fluctuation in the validation set is a lot less dramatic than before and appears to be similar to regular fluctuation I have seen in my experience. Kudos that you have also managed to prevent the model from overfitting.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.