0
$\begingroup$

I am new to clustering.i have data from quality testing of an automobile manufacturing company.

I have 100000 datasets.each dataset has 4 variables force, voltage, current, distance. each variable is a continuous time-series with 8000 data points each(1 to 17000 milliseconds). the length of time series differs from on dataset to another. all variables in one dataset has to be compared with another dataset

I have to find clusters in the 100000 datasets based on similarities in shape of each variable in a dataset.

which type of measuring best suits, in this case, to find similarity in shape of time-series

Improve this question
$\endgroup$
8
  • $\begingroup$ None. You are expecting magic to happen, but it won't. $\endgroup$
    – Has QUIT--Anony-Mousse
    May 17, 2017 at 21:33
  • $\begingroup$ You should maybe do some feature engineering. $\endgroup$
    – Has QUIT--Anony-Mousse
    May 17, 2017 at 21:34
  • $\begingroup$ @Anony-Mousse I think the new edit by OP is answerable? $\endgroup$
    – ABCD
    May 21, 2017 at 10:29
  • 1
    $\begingroup$ @kris your description is quite vague, and the possible correct answers are too many. You should provide more information on the nature of the time series: are the data points categorical, continuous? Do you expect some seasonality or trends? What are the important traits of the time series? What real world stuff is actually represented by the time series (are they day sales, per-minute temperatures)? $\endgroup$
    – noe
    May 21, 2017 at 10:36
  • $\begingroup$ i don't expect seasonality or trend $\endgroup$
    – kris
    May 21, 2017 at 11:02

1 Answer 1

Reset to default
5
$\begingroup$

For most clustering approaches, first you need to choose a similarity measure. Some common default ones for raw time series are Euclidean distance and Dynamic Time Warping (DTW).

When you have computed the similarity measure for every pair of time series, then you can apply hierarchical clustering, k-medoids or any other clustering algorithm that is appropriate for time series (not k-means!, see this).

Update: if the number of time series (along with their size) makes it computationally not acceptable to compute pairwise distances, then one option can be to extract features from each time series, and then use such features as proxies for the time series in the clustering process. Some examples of such features are maximum value, number of peaks, mean value. There are libraries like tsfresh in Python that are meant to easily extract such kind of features from time series. With these features, then any clustering approach like k-means can be applied.

Improve this answer
$\endgroup$
3
  • $\begingroup$ is there any possibility without calculating similarity distance as it becomes more complex for 100000 datasets with four time-series in each dataset. the time-series are not equal in length when compared to time-series of different datasets $\endgroup$
    – kris
    May 21, 2017 at 18:39
  • 1
    $\begingroup$ If performing pair-wise comparisons is a problem, maybe you should do feature extraction and then perform the clustering on those features instead of the original time series. $\endgroup$
    – noe
    May 21, 2017 at 21:12
  • $\begingroup$ can you please provide me more information on clustering after extracting features of time-series. as there are many features for time series can you provide some better features $\endgroup$
    – kris
    May 22, 2017 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.