I have been working on an image classification tasks for which I am extracting the image frames from the video stream collected for different classes.

I have already trained an image classification model (using transfer learning) however due to the outliers (or overlap in the class distribution) accuracy of the model is poor. And not able to generalize the new images / video streams.

Could you please help me with the below queries

  1. How the sample is distributed in each class ? Can I use any visualization techniques (for example : histogram) to see the sample distribution.

  2. And also going through the image one by one is tedious process so is there a technique with which I can find the outliers (outlier images) from the samples. So that I can remove outliers before training the model.

Any updates on this..

Thank you

  • $\begingroup$ What is your kind of outlierness? What does it mean in this specific example to be an outlier as an image? $\endgroup$
    – Leevo
    Commented Jun 18, 2019 at 12:51
  • $\begingroup$ Lets consider I have all dataset of 998 black colored dogs and 2 white colored dogs. If I try to cluster the images, these 2 white colored dogs are going to make a huge difference. So before building the model, I need to make sure all these kinds of outliers are removed from the dataset. $\endgroup$
    – deepguy
    Commented Dec 17, 2019 at 13:55

2 Answers 2


Detecting outliers is actually not an easy task. You can detect outliers by looking at uncertainty measurements. Nevertheless there are different kind of outliers. For example an outlier can be an out of distribution sample (you want to distinguish cats and dogs, but you input a penguin) or you can have "outliers" because the class estimate is not clear (check out chihuahua-or-muffin).

Nevertheless I recommend reading the deep ensembles paper and check out his references. It is more or less easy to follow and they show a method for getting outliers. In essence an outlier will cause disagreement in the model. Modeling the disagreement is the key part of the paper. After getting the disagreement you can look at samples with high disagreement and decide what to do (like adding new classes or removing them, etc..).


BiGAN, Bi-directional GAN(variation of GAN(Generative Adversarial Network)) can be explored for anomaly detection. I have used for my use case. Since I dont have large image dataset, my results are not that good.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.