0
$\begingroup$

I have a set of time series data that I would like to feed into a clustering algorithm (like k-means, using dynamic time warping as the distance function). After standardizing the data with mean 0 and variance 1, the k-means classifier generated a batch of centroids that seemed to fit the data pretty well.

The only question I have is whether the data should be stationary. Models such as ARIMA require for the data to be stationary due to the nature of it. However, the data I want to cluster is mortgage rates as a function of time, which could be subject to seasonal trends, which could be useful when clustering other future time series data.

The question is: do clustering algorithms for time series data generally require for the data to be stationary?

Improve this question
$\endgroup$
6
  • $\begingroup$ What do you mean by standardized - taking out seasonality or scaling all variables to a common range (like from 0 to 1)? $\endgroup$
    – El Burro
    Jun 19, 2019 at 8:43
  • $\begingroup$ @ElBurro oh I'm so sorry, for some reason I used standardization and stationary interchangeably. When I mean that I standardized my data, I mean that I just standardized by making each time series have an overall mean of 0 and a variance of 1. This, however, does not mean that each time series is stationary. Actually, I checked and none of my data, before and after being standardized, are stationary. So my question is whether clustering algorithms for time series, like k-means for instance, require for the data to be stationary. After thinking about it I don't believe it's the case... $\endgroup$
    – Cristian Vives
    Jun 19, 2019 at 18:55
  • $\begingroup$ The reason being that if I'm trying to cluster similar time series, then seasonality can be an important feature to distinguish different time series. I'm not trying to do forecasting, which in that case I know it's necessary for a time series to be stationary. $\endgroup$
    – Cristian Vives
    Jun 19, 2019 at 18:58
  • $\begingroup$ Ok but all dimensions have the same dimensions- which is good as most clustering is essentially n-dimensional distance calculations $\endgroup$
    – El Burro
    Jun 20, 2019 at 16:23
  • $\begingroup$ is your input for the clustering an entire time series or one datapoint (in multiple dimensions)? $\endgroup$
    – El Burro
    Jun 20, 2019 at 16:25

1 Answer 1

Reset to default
0
$\begingroup$

Have you tried replacing the entire timeseries with some meta data of it - clustering on each point of a series seems to be a bit exzessive. Especially as then if your timeseries are not synchronized they might not be considered similar - eventhough they might be for the purpose of this exercise (Imagine two sinus curves but one is slightly phase shifted -- If you would want to cluster by frequency then these would be equal).

So you could try to do something similar to Fourier transformations Fourier Transformations, linear fits, etc and use this meta data as input for the clustering.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Hi El Burro. I was actually applying dynamic time warping as my "distance" calculation for time series. I also standardize each time series with zscore before I begin. DTW is supposed to take care of phase shifts, so do you think this is good enough? $\endgroup$
    – Cristian Vives
    Jun 21, 2019 at 15:12
  • $\begingroup$ Could you maybe post some example code? $\endgroup$
    – El Burro
    Jun 24, 2019 at 11:03
  • $\begingroup$ Sure! Given that the data is represented as an array, where each entry in the array is a time series, I first standardize the array with the following code: [scipy.stats.mstats.zscore(arr,axis=0) for arr in array] After applying zscore standardization, I use the tslearn python module to cluster my data. I use the kmeans for time series clustering algorithm, as seen here: [tslearn.readthedocs.io/en/latest/gen_modules/clustering/… As you can see, there is an option to use DTW as your metric. $\endgroup$
    – Cristian Vives
    Jun 24, 2019 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.