6
$\begingroup$

Been struggling with finding the best approach to handle this scenario, I'm also a novice when it comes to machine learning. I have a dataset of around 700 classes, 90,000 total subclasses, and anywhere from 10 - 1000 images per subclass of the object. For this example, let's say the 1st class is a musical group, and the subclasses are album cover art.

I want to be able to quickly match an image of an album cover to which band and album it belongs to, which architecture can I use to handle this? I am not too familiar with working with such large datasets and this kind of class/subclass structure and am a bit stumped.

Thank you

$\endgroup$
3
  • 1
    $\begingroup$ can u post a few sample images and expected predictions for them? $\endgroup$
    – Justas
    Dec 5, 2022 at 14:02
  • 3
    $\begingroup$ Do you want to classify novel images or is this effectively a lookup scenario where it would be sufficient to say "yup, this is yet another distorted copy of this particular image in our database" ? Those would imply completely different algorithms. $\endgroup$
    – Peteris
    Dec 5, 2022 at 16:24
  • 1
    $\begingroup$ @Peteris yes, ideally the user will scan / photograph their album cover and I want to quickly be able to search through my dataset and find what is the closest match for the cover in the image. From there I can do other things with my application such as show the user similar images of the same cover, pull recent sales history, etc. From research I've done I'm leaning towards maybe using image vector representations of my images and training a CNN that way? $\endgroup$
    – THEOS
    Dec 5, 2022 at 17:49

2 Answers 2

11
$\begingroup$

If you're literally dealing with album covers, then I think trying to train an image classifier might not be the best approach. If you have only 10 examples per subclass (artist), I suspect an image classifier might have a hard time learning a generalizable recognition rule.

I think it might be worth trying OCR on the album cover, and then use text classification on the results of the OCR. With luck, one can hope that the name of the band/artist will be printed on the album cover in text that can be captured via OCR, thus making it significantly easier to infer the band name from the text than from the image.

It also depends on how much you want the ML to generalize. Do you want it to be able to produce the right answer on new albums that it hasn't seen during training, having only seen other albums from the same band? Or do you only need it to handle different scans of the same album cover that was already seen during training? The latter is much easier, and might be able to be handled by a simple image similarity network combined with k-nearest neighbor. The former is much harder.

Finally, as always with any computer vision project, start small. Prototype something, try it on a small subset of the dataset, see how well it works, and analyze what errors it makes. Then use that to iterate. Don't expect that you're going to be able to sit down, start from a three-sentence description of the problem, think about it, and then write down the best solution. Instead, it's likely that you'll need to do a lot of experimentation and iterate based on the results of the experiments.

$\endgroup$
2
  • 3
    $\begingroup$ This is a great answer. Sometimes the right approach with data science is to think about how you can refocus the problem. $\endgroup$ Dec 5, 2022 at 16:11
  • 1
    $\begingroup$ Thank you for your answer, it is much appreciated. I really only need it to do a visual search to find a cover it has seen during training, if it comes across an unknown album cover, I can just have the user handle it manually. I will investigate image similarity + k-nearest neighbor. $\endgroup$
    – THEOS
    Dec 5, 2022 at 20:19
6
$\begingroup$

You have multiple problems. I advise you to break it down and tackle each one at the time.

The first problem is the fact that your dataset seems extremely imbalanced, take a look at the methods of image classification on imbalanced dataset. This post is a good starting point.

For the size of your database, I suggest you use a relatively small size CNN with modern regularization techniques, and also use hyperparameter tuning with Bayesian Optimization. Try this and if it didn't work, try larger models. I say this because training on large dataset gets more expensive with larger models.

Hope this helps.

$\endgroup$
1
  • 2
    $\begingroup$ Thank you for this, I can definitely narrow down some results and balance out my dataset, your link was very insightful. $\endgroup$
    – THEOS
    Dec 5, 2022 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.