1
$\begingroup$

In my ConvNet model, i'm trying to classify some images. It is malware images and it doesn't contain complex features (i think), as expected model learn to classify images easily. You can see my network topology summary here:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_17 (Conv2D)           (None, 128, 128, 32)      1312      
_________________________________________________________________
conv2d_18 (Conv2D)           (None, 125, 119, 32)      40992     
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 62, 59, 32)        0         
_________________________________________________________________
conv2d_19 (Conv2D)           (None, 62, 59, 32)        40992     
_________________________________________________________________
conv2d_20 (Conv2D)           (None, 59, 50, 32)        40992     
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 29, 25, 32)        0         
_________________________________________________________________
flatten_5 (Flatten)          (None, 23200)             0         
_________________________________________________________________
dense_9 (Dense)              (None, 64)                1484800   
_________________________________________________________________
batch_normalization_5 (Batch (None, 64)                256       
_________________________________________________________________
activation_5 (Activation)    (None, 64)                0         
_________________________________________________________________
dropout_5 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_10 (Dense)             (None, 8)                 520       
=================================================================
Total params: 1,609,864
Trainable params: 1,609,736
Non-trainable params: 128
_________________________________________________________________

But, after a while accuracy still improving (slowly) val_accuracy start to oscillate and accuracy difference between training and validate data increase after that. I'm doing cross-validation on dataset with same distribution of all classes. You can see some of the training outputs below:

Epoch 20/80
240/240 [==============================] - 64s 267ms/step - loss: 0.1247 - acc: 0.9654 - val_loss: 0.3417 - val_acc: 0.9270
Epoch 21/80
240/240 [==============================] - 66s 275ms/step - loss: 0.1030 - acc: 0.9700 - val_loss: 0.3560 - val_acc: 0.9220
Epoch 22/80
240/240 [==============================] - 76s 316ms/step - loss: 0.1085 - acc: 0.9671 - val_loss: 0.3471 - val_acc: 0.9100
Epoch 23/80
240/240 [==============================] - 66s 274ms/step - loss: 0.0804 - acc: 0.9787 - val_loss: 0.4013 - val_acc: 0.9060
Epoch 24/80
240/240 [==============================] - 64s 267ms/step - loss: 0.1004 - acc: 0.9725 - val_loss: 0.4071 - val_acc: 0.8920
Epoch 25/80
240/240 [==============================] - 64s 266ms/step - loss: 0.0859 - acc: 0.9754 - val_loss: 0.4733 - val_acc: 0.9110
Epoch 26/80
240/240 [==============================] - 64s 267ms/step - loss: 0.0980 - acc: 0.9717 - val_loss: 0.3792 - val_acc: 0.9120
Epoch 27/80
240/240 [==============================] - 64s 267ms/step - loss: 0.0807 - acc: 0.9775 - val_loss: 0.4354 - val_acc: 0.9100
Epoch 28/80
240/240 [==============================] - 64s 266ms/step - loss: 0.0806 - acc: 0.9754 - val_loss: 0.4109 - val_acc: 0.9070
Epoch 29/80
240/240 [==============================] - 64s 266ms/step - loss: 0.0652 - acc: 0.9821 - val_loss: 0.4318 - val_acc: 0.9070
Epoch 30/80
240/240 [==============================] - 64s 265ms/step - loss: 0.0604 - acc: 0.9825 - val_loss: 0.4095 - val_acc: 0.9190

Is it overfitting or couldn't learn because my net is not deep / neuron count in dense_10 is not enough? I done two or three dense layers after convolutional layers but, it is overfitting after some point.

$\endgroup$
1
$\begingroup$

If the training loss keeps improving while the validation loss stagnates or decreases is a sign of overfitting.

It means that your model is continuing to learn patterns in your training data, so the layer/unit count is certainly not too low. But these patterns are not not general and does not exist in your validation data.

To combat this you can increase regularization, do early stopping or even decrease the amount of units/layers in your network.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.