1
$\begingroup$

I want my machine learning algorithm to learn the difference between two classes, actually picture of X or picture of something else.

My sample data is:

  • 500 pictures of X (I know it is low, unfortunately I can't do much about it)
  • 1,000,000 pictures of something else

Question: Should I do the training with all of the 1,000,000 pictures of something else?
Or would such an imbalance have negative effects? For instance would it kind of "drown" the other data?

Notes:

  • Computing power and time is not an issue.
  • In the real world, pictures of X make up 5% to 10% of the data, so I don't think I have a class imbalance problem.
  • I think the classification is an easy one, machine learning will probably be able to understand quickly.
  • I am OK with a reasonable amount of misclassifications.
  • In case that matters, I am planning to use Keras and Tensorflow with Flatten/Dense relu/Dense softmax/AdamOptimizer/sparse_categorical_crossentropy.
Improve this question
$\endgroup$
5
  • $\begingroup$ The figure is too high! You need to get more data or use heavy augmentations.. Even then the model will be better predicting the major class .... $\endgroup$
    – Aditya
    Commented Sep 20, 2018 at 3:18
  • $\begingroup$ @Aditya: As stated in the question, there is no more data for the first class. That is the premise of the question. You say 1,000,000 is too high, right? Do you suggest I randomly sub-sample? If yes, what number sounds right to you? $\endgroup$
    – Nicolas Raoul
    Commented Sep 20, 2018 at 3:30
  • $\begingroup$ I believe that you can go with 500 Vs 10000-30000 easily with augmentations, but that also depends how accurate you want the model to be, Are the images of the major class varying greatly(if yes, then the selection will be done selectively), what is it going to be used for, Also we really need to try oversampling for the minor lcass and under sampling for the major, create multiple copies of the minor class(simply duplicating will also help, via adding rotations(+-x degrees) of the image by a script(comes under augmentation but we can add it manually to improve count ).PS 500 isn't 5% of 1e6 $\endgroup$
    – Aditya
    Commented Sep 20, 2018 at 3:43
  • $\begingroup$ @Aditya: About the PS: real-world data proportions are different from proportions of the sample I have access to. $\endgroup$
    – Nicolas Raoul
    Commented Sep 21, 2018 at 8:17
  • $\begingroup$ One way to handle such imbalance classes is to downsample the majority class. $\endgroup$
    – Siddhi Kiran Bajracharya
    Commented Nov 20, 2018 at 5:41

1 Answer 1

Reset to default
1
$\begingroup$

Yes, train with all the data.

To adjust for the imbalanced data size, you want your minority class X to be more important than the majority class. You can either (1) use weighted cross-entropy loss, e.g. tf.nn.weighted_cross_entropy_with_logits, or (2) when adding up the loss from different traning examples, weight them inversely proportional to the size of that label, or use both.

This is a good explanation here.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Yep playing with metric is very crucial here! Weighing the lesser class is definitely a good go.. Also have a look at Kappa Metric(Cohen's Kappa):( Classification accuracy normalized by the imbalance of the classes in the data) $\endgroup$
    – Aditya
    Commented Sep 20, 2018 at 3:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.