0
$\begingroup$

I am training a deep learning model for traffic prediction. When I use 10 months of data for training (validation split: 10%) and 2 months for testing. The loss curve looks like this: 10 months training data. and the performance of model is very bad (rmse: 41.98). But when I reduce the training data and use 9 months of data (validation split: 10%) and increase the test size (3 months), the performance is excellent (rmse: 7.92). The loss curve looks like this: enter image description here

In the first training, the training data is large and test data is small. Counterintuitively, it performed poor compared to second training where training data is small and test size is higher.

Can somebody point out what is happening is these two trainings? Also, what these two learning curves tells about the learning? Also, in second training, there is a sudden spike after 20 epochs and steep decline between 120 and 140 epochs. What is happening in these points?

Other parameters:

  • learning rate: 1e-4
  • early stop patience: 30
  • Optimizer: Adam
Improve this question
$\endgroup$
4
  • $\begingroup$ Is it a feedfoward neural network ? How many observations do you have ? And what is the architecture of your neural network ? $\endgroup$
    – Theudbald
    Aug 2 at 15:00
  • $\begingroup$ Its a feed forward network. Here is the link: . Around 8000 training examples and 1500 test examples. $\endgroup$
    – user2380384
    Aug 2 at 15:21
  • $\begingroup$ Does this happen irrespective of which 2 (or 3) months you use as the test data? $\endgroup$
    – Lynn
    Aug 6 at 9:59
  • $\begingroup$ Good point. When I change the months of test data, the graph looks more or less like the first graph. Any insights would be helpful. $\endgroup$
    – user2380384
    Aug 9 at 13:15

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy