# RGB Image Segmentation using Clustering

I want to apply some segmentation on a dataset for preprocessing purposes. I have tried the "otsu thresholding" approach in order to segment the image. It's a good method, however, I think a clustering algorithm such as K-Means can be even more succesful for the basic segmentation. Here is some problem with Otsu Thresholding:

As you can see, in some samples, Otsu thresholding may fail like in the picture 1. I think that, using a pixel-wise positioning approach as well as color thresholding may increase the success even further.

However, I couldn't manage to work with scikit-learn's KMeans function to work with an RGB image, since it is a 3 channel 2-d matrix(basically 3d matrix).

How can I combine both pixel-wise distance approach as well as color clustering with K-Means clustering or alike method?

I am not sure about pixelwise distance but what I could help is on applying KMeans on this picture. Let's say I give you this picture (I cannot get your original image so I'll just use mine).

Implementing KMeans on image is actually quite straightforward. What you might want to pay attention to is to the size of the image since big image like I what I am giving you here might cause many problems, such as big memory consumption.

The trick here is to resize image during training, you can use full image during prediction. You can think of it like this, scaling the image will roughly preserve how colors are distributed over the whole image so scaling down first will not change much on the cluster centres found by KMeans.

To do clustering, simply stack the image to 2D array and fit KMeans over this since we only cluster with pixel values. To get the segmented (clustered image) simply extract the cluster centres, replace the cluster with its respective centre and then rearrange back to the original shape. The code to do that is provided below.

img = cv2.imread('photo-1436891436013-5965265af5fc.jpg') ## Simply replace with your image file
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (int(img.shape[1]/6),int(img.shape[0]/6)))

n_clust=5
km = KMeans(n_clust, n_jobs=-1)
clust = km.fit_predict(img_scaled)
ctrs = km.cluster_centers_
ctrs_map = dict(zip(np.arange(n_clust),ctrs))
segmented = np.array([ctrs_map[t] for t in clust]).reshape(img.shape[0],img.shape[1],3) * 255.0
segmented = segmented.astype(int)
plt.imshow(segmented)


Now you can also visualize for different K (I am not providing the code as it is quite trivial given the code above).

You will want to implement this yourself.

Encoding the data as 5 dimensional vectors per pixel crates a substantial memory overhead. But if you can try that first, to see if the results are as good as you believe then to be... Pay attention to scaling!