0
$\begingroup$

I am trying to think through my process before doing any real coding. However, got really confused easily.

Say I have 100 instruments and I know their price movements every day for a year. So I can create a movement matrix 

A =[[I1-1, I2-1, .... I100-1],  (I1-1 is price for instrument 1 on day 1)
    [I1-2, I2-2, .... I100-2],
    ....
    [I1-365, I-2365, .... I100-365]
    ]

Then for each instrument, I can calculate a price movement correlation between other instruments for the whole year.

   C =[C1-2, C1-3,...C1-100,C2-3,....C99-100] (C1-2 is the price movement correlation between instrument 1 and 2 for the whole year)

Then I would like to apply a K-Means clustering algorithm to classify the correlation into say 10 categories. So in theory, I created 10 categories that the prices turned to move together.

However, the more I think about it, the more it is not correct. For example, if this is my Correlation result:

 C =[0.35, 0.59,...0.88(C1-100),0.48,....0.99(C99-100)]

isn't it K-Means clustering may classify C1-100, C99-100 in one cluster, and C1-2, C1-3, C2-3 in another cluster.

When I read that, it means instrument 1,100, 99 in one category, and instrument 1,2,3 in another category. But I would like each instrument only available in one category, so looks like there is a hole in my idea or maybe my idea is totally wrong?

Improve this question
$\endgroup$
2
  • $\begingroup$ K-means does not accept a distance matrix. It must be used on the raw data in order to compute the means, and it will always compute the squared Euclidean distances - it cannot optimize arbitrary distance functions! $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Oct 31, 2018 at 14:42
  • $\begingroup$ not understanding why you are needing kmeans to do this if you only have 1 metric, price movement. kmeans needs more than 1 feature. $\endgroup$
    – Ralph Winters
    Commented May 21, 2022 at 21:06

1 Answer 1

Reset to default
0
$\begingroup$

You will not get what you seek this way, but you are on the right path. Use the correlation between two instruments as a measure of similarity, and then perform spectral clustering with this measure as the kernel.

Basically, you will start with your correlation matrix $R$, and build the corresponding Laplacian matrix $L$. The eigen vectors corresponding to the smallest eigen values of $L$ will give you a projection space in which you can perform k-means clustering.

This technique is efficient if you have a good similarity measure, but only works for reasonably sized datasets (because you need the eigen decomposition of the $n \times n$ matrix).

Improve this answer
$\endgroup$
2
  • $\begingroup$ sorry, I am not familiar with spectral clustering and just looking into it. Are you saying I should do k mean clustering after spectral clustering or spectral clustering is the tool I should go for (so no need for k means). $\endgroup$
    – daxu
    Commented Oct 31, 2018 at 11:47
  • $\begingroup$ Spectral clustering means making the projection (the way explained above) and perform a standard clustering method (such as k-means). This is basically just a trick to make k-means work when euclidian (standard) k-means can't work because the euclidian distance does not represent the problem well enough. $\endgroup$
    – Romain Reboulleau
    Commented Nov 1, 2018 at 15:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.