I am trying to train a neural network with 2 hidden layers to perform a multi class classification of 3 different classes. There is a huge imbalance to the classes, with the distribution being around 80%, 13% and 7% for class 1, 2, and 3.

I am facing the issue where my training loss is much higher than my validation loss and my training accuracy is lower than validation accuracy. Moreover, the shapes of my training and validation loss curves are a little strange.

enter image description here

enter image description here

The data set I am using is not small, with around 200k rows of training data and 50k validation data. Currently, I am using a learning rate of 0.01, with 40 epochs and 50 nodes within each hidden layer.

Have been stuck on this for a while and not too sure where I am going wrong. Appreciate any help, thanks!

  • $\begingroup$ Hi @josephwong, welcome to the site. In your model, are you using dropout, batch normalization or any other element that changes its behaviour between training and inference modes? $\endgroup$
    – noe
    Commented Sep 18, 2023 at 19:06
  • $\begingroup$ Hi Noe, thanks! Nope i am not using any dropout or regularization, though I am using early stopping. $\endgroup$ Commented Sep 19, 2023 at 2:51


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.