EfficientNet model was trained on ~3500 images for a 4-class classification: A, B, C and Neither – with accuracy of 0.985 – by someone else, not me. I'm quite new to ML. So we have this model, and it works pretty well.

As more real-world images are coming in from the users, we see more errors. Is it possible to improve the performance a bit by adding more training examples?

Collected and manually sorted 35 images of class A and 425 of class "Neither". Does it even make any sense with so small dataset?

I have put images into a folder with 4 subfolders, one per each class. Two are empty, two have images. Using Keras ImageDataGenerator flow_from_directory() method with validation/training = 0.2/0.8 ratio.

model.fit() training shows low loss and accuracy close to 1, while validation loss is high and accuracy is around 0.21 — 0.25 only.

Model performance after my "training" is totaly ruined.

Is this approach to retraining with a little set of images wrong?

How can I "add", not "replace" the model "skills" with a new added little set of images?

Maybe I should have trained a fresh model initialised with the ImageNet weights as in the initial training, and just add the new images to the set of images used, again, in the initial training?


1 Answer 1


The problem you describe is called transfer learning, which uses uses a pretrained model (such as EfficientNet) that is trained on an existing dataset and applies it to a different dataset. Since this method uses a pretrained model, the new dataset on which the transfer learning needs to happen doesn't need the same number of training examples compared to when the model would be trained from scratch.

In your specific case, you could look into using an existing EfficientNet model that is pretrained of a different dataset (such as ImageNet) and add one or two layers to make the model specific for your dataset (i.e. number of output neurons equal to the number of classes you are trying to classify). You can then train the pretrained model on your specific dataset, however, make sure to start by fixing the parameters of the pretrained model to make sure the model keeps the knowledge it has learnt.

  • $\begingroup$ That was already done: I have the model that was initialized from ImageNet weights and then trained on 3500 custom images for a 4-class classification. It works, and works pretty well. I wonder how can I improve that even more, given a little additional set of images to train on. $\endgroup$
    – Serge
    Feb 24, 2020 at 20:07
  • $\begingroup$ The process would be the same, however given the extra data that you collected I am not sure how much that would improve the performance, but you can always try. $\endgroup$
    – Oxbowerce
    Feb 24, 2020 at 20:24
  • $\begingroup$ My question is which path should work: a) take current working weights and train on just the new 500 images; b) train from zero (ImageNet weights) on 3500 initial images + 500 new ones? $\endgroup$
    – Serge
    Feb 24, 2020 at 20:29
  • $\begingroup$ I would probably choose option b given that the total size of your dataset is not that big. $\endgroup$
    – Oxbowerce
    Feb 24, 2020 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.