I'm trying to retrain Inception model final layer for a binary classification.
My training image set contain 2000 images in class 1 and more than 6000 images in class 2.
Will this huge difference in number of images of each class in training set affect my classification?
The short answer is yes.
While your classes are imbalanced, model will be more likely to "learn" class 2 during the training, especially when it comes to mini-batch updates. (even though each mini-batch is unlikely to have only class 2 samples)
The common solution is to use weighted loss function or feed weighted samples to mini-batch.
In my practice, I always use weighted samples if I have sufficient data, even when my classes are not that skewed. Weights always help, at least it never ruins the model.
