1
$\begingroup$

I trained my data over 40 epochs but got finally this shape. How can I deal with this problem? Please as I used 30.000 for training and 5000 for testing and

lr_schedule = keras.optimizers.schedules.ExponentialDecay(
    initial_learning_rate=4e-4,
    decay_steps=50000,
    decay_rate=0.5)

enter image description here

should I increase the number of data in testing or make changes in the model?

EDIT

After I add regularization I got this shape and the loss started from a number greater than before in the previous shape, does that normal?

enter image description here

Is this good training or is there still a problem?

$\endgroup$
7
  • $\begingroup$ Are you sure you are overfitting? Typically when overfitting the training loss will be going down, while validation loss will be going up. Your case actually looks like underfitting to me. $\endgroup$
    – Akavall
    May 18, 2022 at 5:08
  • $\begingroup$ so any gap i got between train and test called underfitting as I'm not sure $\endgroup$
    – Lei
    May 18, 2022 at 5:44
  • $\begingroup$ Try to normalize the data and try Regulation methods $\endgroup$ May 18, 2022 at 7:59
  • $\begingroup$ It's also possible that the model is fine actually. The Y axis shows the range 4.4 to 5.6, so the curves look distant from each other. If plotted on a range of 0 to 10 for instance they would look not very far from each other. $\endgroup$
    – Erwan
    May 18, 2022 at 9:07
  • $\begingroup$ @Erwan excuse me can you please see the edited post as I added regularization to the model and got the shape I posted but the accuracy is still low .. what are the factor that affect on the accuracy $\endgroup$
    – Lei
    May 19, 2022 at 4:05

1 Answer 1

2
$\begingroup$

Here are some proposal. I would need to see the code to be more specific.

Did you randomize your data and split to train and validation parts?

Have you applied any dropout to your learning process?

Did you normalize the data?

It seems that your model use quite different set of data, having them randomly organized could solve your issue. On the other hand, a 10% drop out could often avoid overfitting issues because it resets a part of neural network weights. Lack of normalization could also block the neurons to specific ranges of data and explain bad results in the validation dataset.

$\endgroup$
2
  • $\begingroup$ Thanks a lot. I'm already splitting the data into train and test and added dropout but excuse me what do you mean by normalise the data ,please $\endgroup$
    – Lei
    May 18, 2022 at 10:18
  • 1
    $\begingroup$ It means divide all the data by the maximum possible value in order to have a range between 0 and 1. I don't know the kind of NN you are using, but it is often safer to have the data normalized in order to avoid weights mistakes due to their initial weights that can't learn in some cases. $\endgroup$ May 18, 2022 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.