# Validation loss much higher than training loss

I am training a CNN on some text data. The sentences are padded and embedded and fed to a CNN. The model architecture is:

model = Sequential()

loss='binary_crossentropy',
metrics=['accuracy'])


Any help would be appreciated.

Your model is completely overfitting. The training loss is constantly decreasing but the validation loss isn't. This means that the your current model is complex enough to 'memorize' the patterns in the training data. In such situations, you need to regularize your model.

To regularize your data you may try any or all combination of the following:

1) Reduce the layers of the neural network.

2) Reduce the number of neurons in each layer of the network to reduce the number of parameters.

3) Add dropout and tune its rate.

4) Use L2 normalisation on the parameter weights and tune the lambda value.

5) If possible add more data for training.

6) Adding more data may not always be economically viable option so 'Data Augmentation' can play a big role here. Data Augmentation depends on the type of input to the neural nets. For images they comprise of rotation, translation, affine transformation like shear etc.

• For text data, a commonly used (with great impact) method of reducing overfitting by bringing in extra data is using word (or subword) embeddings that are pre-trained on a very large unlabeled corpus. Word2vec, fasttext, BERT and friends, etc are all variations on that concept. One could even argue that replacing words with pre-trained embeddings is something like the linguistic equivalent of the image augmentation methods. Dec 27 '19 at 2:26

You are over-fitting to the train dataset and failing to generalize to the validation dataset.

One way to reduce overfitting is regularization. The most common methods for regularizing a neural network are: