# confusing regarding to kmeans clulstering for data correlation

I am trying to think through my process before doing any real coding. However, got really confused easily.

Say I have 100 instruments and I know their price movements every day for a year. So I can create a movement matrix

A =[[I1-1, I2-1, .... I100-1],  (I1-1 is price for instrument 1 on day 1)
[I1-2, I2-2, .... I100-2],
....
[I1-365, I-2365, .... I100-365]
]


Then for each instrument, I can calculate a price movement correlation between other instruments for the whole year.

   C =[C1-2, C1-3,...C1-100,C2-3,....C99-100] (C1-2 is the price movement correlation between instrument 1 and 2 for the whole year)


Then I would like to apply a K-Means clustering algorithm to classify the correlation into say 10 categories. So in theory, I created 10 categories that the prices turned to move together.

However, the more I think about it, the more it is not correct. For example, if this is my Correlation result:

 C =[0.35, 0.59,...0.88(C1-100),0.48,....0.99(C99-100)]


isn't it K-Means clustering may classify C1-100, C99-100 in one cluster, and C1-2, C1-3, C2-3 in another cluster.

When I read that, it means instrument 1,100, 99 in one category, and instrument 1,2,3 in another category. But I would like each instrument only available in one category, so looks like there is a hole in my idea or maybe my idea is totally wrong?

• K-means does not accept a distance matrix. It must be used on the raw data in order to compute the means, and it will always compute the squared Euclidean distances - it cannot optimize arbitrary distance functions! Commented Oct 31, 2018 at 14:42
• not understanding why you are needing kmeans to do this if you only have 1 metric, price movement. kmeans needs more than 1 feature. Commented May 21, 2022 at 21:06

Basically, you will start with your correlation matrix $$R$$, and build the corresponding Laplacian matrix $$L$$. The eigen vectors corresponding to the smallest eigen values of $$L$$ will give you a projection space in which you can perform k-means clustering.
This technique is efficient if you have a good similarity measure, but only works for reasonably sized datasets (because you need the eigen decomposition of the $$n \times n$$ matrix).