0
$\begingroup$

I have tried adding a layer in the Resnet Model as shown:

fuente ="https://tfhub.dev/tensorflow/resnet_50/feature_vector/1"
class ResNetModel(tf.keras.Model):
    def __init__(self, classes):
        super(ResNetModel, self).__init__()        
        self._feature_extractor = hub.KerasLayer(fuente, trainable=False) 
        self._added_layer = tf.keras.layers.Dense(32, activation = "relu",trainable = True)
        self._classifier = tf.keras.layers.Dense(classes, activation='softmax')

    def call(self, inputs):
        x = self._feature_extractor(inputs)
        x= self._added_layer(x)
        x = self._classifier(x)
        return x

The test accuracy has diminished a lot. When training with more epochs, the training accuracy rapidly arrives at 100%, but the test accuracy stucks at 61.7%, much worse that using the model without the added layer(72% with 10 epochs). I expected some improvement in the result. Where is my error?

$\endgroup$
4
  • $\begingroup$ Your model is overfitting after adding a layer. What was the train score w/o the extra layer? $\endgroup$
    – Jonathan
    Sep 19 at 12:54
  • $\begingroup$ Without the layer (10 epochs) test accuracy 71%.. Whith the _added_layer (100 epochs) test accuracy 61% $\endgroup$ Sep 19 at 13:03
  • $\begingroup$ Can you also share the score on train data for your model without the extra layer? Would be even better to see the learning curves for both models (i.e. #epochs vs. loss for both models on train and test data). $\endgroup$
    – Jonathan
    Sep 19 at 15:44
  • $\begingroup$ Accuracy on train data is very high in both cases. Without the _added_layer in 10 epochs is about 98%, and with the added layer 100% $\endgroup$ Sep 19 at 15:55

1 Answer 1

1
$\begingroup$

Both of your models (with and with no extra layer) have high train accuracy but much lower scores on test data which indicates that both models overfit, i.e. they suffer from high variance (using the bias-variance trade-off terminology). In essence, it means that your models perform poorly on unseen data.

To gain a better understanding of what is happening, I'd plot the learning curves of your model with no extra layer (see here for an explanation of and links to further reads on learning curves).

Generally, instead of adding an extra layer (and thereby increasing model capacity) you rather need to do the opposite in case of overfitting, e.g. by

  • reducing model complexity (e.g. train for less epochs) or
  • increasing the dataset size (e.g. by data augmentation).

The Deep Learning book by Ian Goodfellow provides some helpful practical recommendations in this chapter.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.