4
$\begingroup$

I've got a collection of yearly data (one value per year per category), and I'd like to find series that are most similar to one another.

Example data is here.

I don't know much about data science, but it seems like cosine similarity might be the way to go? If so, how do I account for the nil values in the datasets?

Improve this question
$\endgroup$
1
  • $\begingroup$ For clustering time series, you can usually use some dynamic time warping based approaches. For further readings I would redirect to this $\endgroup$
    – nyro_0
    Commented Mar 20, 2018 at 9:15

1 Answer 1

Reset to default
4
$\begingroup$

Since the time-series are annual, the data points you have for each time-series are limited and also quite distant (the values are 1 year apart). So I wouldn't use Dynamic Time Wrapping on your data.

If you are interested in comparing the patterns, a very simple approach would be Pearson's correlation. Keep in mind that this will not compare the actual values but the patterns (i.e. if the values have similar fluctuations with the years, so for example time-series [1 2 3 4] would have higher correlation with [5 6 7 8] than with [1 1 2 2])

If you are interested in both values and pattern, I would use a distance-based metric: Euclidean Distance, Manhattan distance etc. I believe you will find this post interesting, where the mathematical background of similarity is explained. Also, python implementations of several distance metrics in python (including cosine-similarity) can be found in this blog-post.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.