I am looking for general image-based clustering methods

Question

My task is to cluster some images, I decided to use the VGG model to extract the features and then use K-Means to cluster these features.

But my question: When I use a VGG as a feature extractor, I should make sure if the VGG model was trained on this type of images before, otherwise, the VGG model is not generalizable to all types of images, am I right?

I am looking for a general method to cluster images regardless of the type of dataset efficiently. If you know any efficient image-based clustering methods, could you please point them out?

Thank you in advance.

Answer 1

I would argue, that actually you may have better generalization if you are not training on your images at all.

If you are getting acceptable clustering results using generic VGG features (trained on ImageNet) on your specific task, then you have the best chance that it would work on a wide variety of images.

However, if you need some sort of training on your images to get good results in your tests, then you always risk that if your dataset is not big and diverse enough, you will end up overfitting, and deteriorate generalization.

I personally ran into situations, when an untrained VGG model worked better on new images, than a model finetuned on a small dataset, that ended up overfitting.