0
$\begingroup$

I have a set of time series data that I would like to feed into a clustering algorithm (like k-means, using dynamic time warping as the distance function). After standardizing the data with mean 0 and variance 1, the k-means classifier generated a batch of centroids that seemed to fit the data pretty well.

The only question I have is whether the data should be stationary. Models such as ARIMA require for the data to be stationary due to the nature of it. However, the data I want to cluster is mortgage rates as a function of time, which could be subject to seasonal trends, which could be useful when clustering other future time series data.

The question is: do clustering algorithms for time series data generally require for the data to be stationary?

$\endgroup$
  • $\begingroup$ What do you mean by standardized - taking out seasonality or scaling all variables to a common range (like from 0 to 1)? $\endgroup$ – El Burro Jun 19 '19 at 8:43
  • $\begingroup$ @ElBurro oh I'm so sorry, for some reason I used standardization and stationary interchangeably. When I mean that I standardized my data, I mean that I just standardized by making each time series have an overall mean of 0 and a variance of 1. This, however, does not mean that each time series is stationary. Actually, I checked and none of my data, before and after being standardized, are stationary. So my question is whether clustering algorithms for time series, like k-means for instance, require for the data to be stationary. After thinking about it I don't believe it's the case... $\endgroup$ – Cristian Vives Jun 19 '19 at 18:55
  • $\begingroup$ The reason being that if I'm trying to cluster similar time series, then seasonality can be an important feature to distinguish different time series. I'm not trying to do forecasting, which in that case I know it's necessary for a time series to be stationary. $\endgroup$ – Cristian Vives Jun 19 '19 at 18:58
  • $\begingroup$ Ok but all dimensions have the same dimensions- which is good as most clustering is essentially n-dimensional distance calculations $\endgroup$ – El Burro Jun 20 '19 at 16:23
  • $\begingroup$ is your input for the clustering an entire time series or one datapoint (in multiple dimensions)? $\endgroup$ – El Burro Jun 20 '19 at 16:25
0
$\begingroup$

Have you tried replacing the entire timeseries with some meta data of it - clustering on each point of a series seems to be a bit exzessive. Especially as then if your timeseries are not synchronized they might not be considered similar - eventhough they might be for the purpose of this exercise (Imagine two sinus curves but one is slightly phase shifted -- If you would want to cluster by frequency then these would be equal).

So you could try to do something similar to Fourier transformations Fourier Transformations, linear fits, etc and use this meta data as input for the clustering.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Hi El Burro. I was actually applying dynamic time warping as my "distance" calculation for time series. I also standardize each time series with zscore before I begin. DTW is supposed to take care of phase shifts, so do you think this is good enough? $\endgroup$ – Cristian Vives Jun 21 '19 at 15:12
  • $\begingroup$ Could you maybe post some example code? $\endgroup$ – El Burro Jun 24 '19 at 11:03
  • $\begingroup$ Sure! Given that the data is represented as an array, where each entry in the array is a time series, I first standardize the array with the following code: [scipy.stats.mstats.zscore(arr,axis=0) for arr in array] After applying zscore standardization, I use the tslearn python module to cluster my data. I use the kmeans for time series clustering algorithm, as seen here: [tslearn.readthedocs.io/en/latest/gen_modules/clustering/… As you can see, there is an option to use DTW as your metric. $\endgroup$ – Cristian Vives Jun 24 '19 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.