Image classification architecture for dataset with 710 classes, 90,000 subclasses, and anywhere from 10-1000 images per subclass?

Question

Been struggling with finding the best approach to handle this scenario, I'm also a novice when it comes to machine learning. I have a dataset of around 700 classes, 90,000 total subclasses, and anywhere from 10 - 1000 images per subclass of the object. For this example, let's say the 1st class is a musical group, and the subclasses are album cover art.

I want to be able to quickly match an image of an album cover to which band and album it belongs to, which architecture can I use to handle this? I am not too familiar with working with such large datasets and this kind of class/subclass structure and am a bit stumped.

Thank you

Answer 1

If you're literally dealing with album covers, then I think trying to train an image classifier might not be the best approach. If you have only 10 examples per subclass (artist), I suspect an image classifier might have a hard time learning a generalizable recognition rule.

I think it might be worth trying OCR on the album cover, and then use text classification on the results of the OCR. With luck, one can hope that the name of the band/artist will be printed on the album cover in text that can be captured via OCR, thus making it significantly easier to infer the band name from the text than from the image.

It also depends on how much you want the ML to generalize. Do you want it to be able to produce the right answer on new albums that it hasn't seen during training, having only seen other albums from the same band? Or do you only need it to handle different scans of the same album cover that was already seen during training? The latter is much easier, and might be able to be handled by a simple image similarity network combined with k-nearest neighbor. The former is much harder.

Finally, as always with any computer vision project, start small. Prototype something, try it on a small subset of the dataset, see how well it works, and analyze what errors it makes. Then use that to iterate. Don't expect that you're going to be able to sit down, start from a three-sentence description of the problem, think about it, and then write down the best solution. Instead, it's likely that you'll need to do a lot of experimentation and iterate based on the results of the experiments.

Answer 2

You have multiple problems. I advise you to break it down and tackle each one at the time.

The first problem is the fact that your dataset seems extremely imbalanced, take a look at the methods of image classification on imbalanced dataset. This post is a good starting point.

For the size of your database, I suggest you use a relatively small size CNN with modern regularization techniques, and also use hyperparameter tuning with Bayesian Optimization. Try this and if it didn't work, try larger models. I say this because training on large dataset gets more expensive with larger models.

Hope this helps.