I have 32k black and white images. Want to do clustering on them

Question

As the title says I'm trying to do clustering on a set of black and white images. These images are all 200x200 with black dots on a white canvas Example pics here (These are not actual photos from the data set. Just a general idea of what they look like). The idea is to hopefully find an underlying pattern when it comes to general shape of these images and hopefully cluster by that.

What I've done so far is turned each of my 200 x 200 image in a numpy array of size 40,000. Then put all the images together into a singly numpy array of 32k x 40k.

I'm kind of not sure where to go from there. What I did next was use scikit learn's TruncateSVD on my data set and set the paramater 'n_componenets=100' and fitted and transformed it, so now my data set of images are 32k x 100.

From here on out I don't know what's the best plan of action. Should I just start using k_means algos on my data set? and how would I visualize it?

Sorry if this is a lot. Any help/tips would be much appreciated it.

Answer 1

As the dimension of your data is very high (40000), you can try using Spherical K-Means on your raw data. It uses the cosine similarity to compute the distance between 2 points. Indeed, euclidian distance is not so much informative in high dimensional vector spaces, because of the curse of dimensionality. see https://stats.stackexchange.com/questions/99171/why-is-euclidean-distance-not-a-good-metric-in-high-dimensions

Or you can apply a dimension reduction strategy like truncated SVD or ACP like you did, and then apply the classical k-means algorithm. So yeah you should try to apply k-means on your 100-dimensions data. Maybe try with k = 30 or something similar as 100 may already be too high.

Answer 2

For image clustering, I don't suggest you to start with clustering techniques immediately. First, run some dimensionality reduction model.

The most powerful dimensionality reduction technique is represented by Autoencoders. If you are familiar with Deep learning, use Autoencoders to generate compressed, dense representations of the images. Alternatively, you can use "lighter" models, such as PCA, kernel-PCA, or some manifold learning technique.

This first step has the advantage of reducing noise, multicollinearity, and sparsity in your data. Most importantly, it can save you from the curse of dimensionality.

Once you have compressed the original data, then I suggest you to run k-Means clustering (or another technique if you prefer). Its results will be much more meaningful and easy to interepret.