Multivariate Time-Series Clustering

Question

I have a streaming data along with timestamp dataset that looks like this: 1.png

Timestamp can be inclusive of "seconds" too, but the data may or may not change every second. it depends on the previous values(rows i.e data which came earlier w.r.t time).

Column1, Column2 .... ColumnN correspond to the variables (they change over time) and "Label" shows the different samples. You can assume that the values tend to decrease over time for a particular label.

Labels A1,B1,C1.........A2,... M labels.

Note : Values of timeNew of a Label depends on values of timeOld of that Label and Labels belong to its cluster.

I need to group Labels with similar behavior over time together (e.g. Label A1 and Label C1 should be put in the same cluster and B1,D2 may fall into same cluster over time as they tend to behave similar over time).

I thought of using DTW and get the similarity of each Label with respect to other Labels. but not sure, how to proceed when i have N Columns.

To be precise, i need to group Labels based on their similarities (Column1 .. ColumnN) over time and group them.

Once i group them when new data comes in i should be able to predict the values(Column1.. ColumnN) for a Label based on the previously seen data(can be just minutes closer to the current prediction) and the values associated with the Labels in its cluster and predict it accordingly.

Answer 1

Aleksandr Blekh's answer in this older question provides a lot of interesting reading material for time-series clustering methods and examples. Also, I include below some interesting reading material for calculating similarity among multivariate time-series (the latest 2 are quite old but I think they are very interesting):

• An approach on the use of DTW with multivariate time-series (the paper actual refers to classification but you might want to use the idea and adjust it for clustering)

• A paper on clustering of time-series

• A PCA-based similarity measure for multivariate time-series

• A review on feature extraction and pattern recognition methods in time-series data

Before proceeding with any method, I believe it is important to spend some time to think of the following:

• Try to select the right step for your input data (e.g. if the time-steps are per second, the time-series might be too long and unnecessarily detailed for this job, while hourly data might catch the patterns better).

• Seasonality might be interesting to take under consideration: e.g. if the time-series are hourly and last for several days/months, there might be some daily/monthly seasonality. In this case, you might want to calculate and compare the average day/month from each time-series (if so, you will also need to decide whether weekdays and weekends should be averaged all together or treated in a different way).

• Depending on what you are trying to find out, you need to decide whether Dynamic time warping (DTW) is useful for you. For example, if 2 time-series have the exact same pattern but one of the 2 has a time-delay, should they still belong in the same cluster? (and how small/big time-delay is acceptable to put them in the same cluster?)

Answer 2

DTW per-group is the obvious answer. I've found DTW to be extremely computationally expensive, however.

My favorite technique in the world is lesser-known/used, and would handle your problem nicely, however... and have the added benefit of being scalable.

The downside to this technique is that it would only cluster on the "shape" of the time series, and not require it to line up at a specific time interval like DTW will.

It's called SAX. Basically, you represent the time series as a string of letters. Then, you can treat this string of letters exactly like you do in NLP or text-mining - by creating a frequency matrix for each letter, n-grams, etc. Now, along with all of those features you could add your other features as well, and run normal dimensionality reduction and clustering.

Answer 3

It depends a bit if the timestamps have any connection to each other (is t2 impacted by t1 as example).

In general this looks like a classification problem and you can use for example sklearn. If you want to distinguish A and C and all other cases you would end up with a multi-class classification problem and not all algorithms support these. If not You can just transform the Label into target (A+B) /non_target (the rest).

Three additional advice:

1. You can use pandas Dataframes for the pre-processing.
2. You should at least have a basic understanding of test and training samples before you start anything.
3. The correct success metric is both important and sometimes challenging to find. Quick example: If you choose accuracy and 99% of your values belong to a class any algorithm that per default predicts this class will have a very high accuracy.

If timestamps are connected you might want to do some pre-processing and add data from previous timestamps to the current one (example: average value of column 1 of last x previous tstamps).