0
$\begingroup$

What I met a problem is I do time-series clustering, and I found the clustering result isn't ideal.

I can't use elbow method to know what clustering result is good, that means I have no ways to watch the clustering results and tune parameters.

What I want to do is cluster the same trends data into the same group.

But I met a problem like below:

>> arr # It's a time-series array with 10 time intervals.
array([[  0,   0,   100, 0,   0,   0,   0,   0,   0,   0],
       [  0,   0,   0,   0,   0,   0,   0,   0, 100,   0],
       [100,   0,   0,   0,   0,   0,   0,   0,   0,   0]])

>> model = KM(n_clusters=2).fit(arr) ; model.labels_
array([1, 0, 0], dtype=int32)

It clusters arr[1] and arr[2] the same group!

But!With our eye watching, we all know arr[0] and arr[2] should be the same group because they have 100 in the near time intervals.

How to do time-points clustering? And specify random_state=N is useless becuase there is always one corner case to let me fail.

UPDATE:

There's one algorithm to solve this and its name is KShape. It can be found in tslearn github.

It clusters time-series data based on the shape of each data, so it match my needs.

$\endgroup$
2
$\begingroup$

In k-means there is no similarity or order of axes. They are all assumed to be independent. Hence, these three series are all equally dissimilar. You got what you asked for - but the question you asked (minimize the squared errors in each component) was not what you wanted.

You'll need to choose a different algorithm. One that knows about time, and allows for some tolerance in time, or one where you can use a distance function such as DTW to measure similarity.

Don't treat clustering as a black box. That will not work, you'll get results that you don't like. Instead, understand the algorithms first, then guide them carefully to solve your problem.

$\endgroup$
  • $\begingroup$ +1 for understanding the question, I got it after reading your answer! $\endgroup$ – Erwan Jun 15 '19 at 18:19
  • $\begingroup$ BTW, There's a tool to solve this, it's KShape and it can be found in tslearn github. It cluster by the shape of time-series data. $\endgroup$ – code_worker Jun 18 '19 at 2:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.