Considering there are two series over time and new data is added in the series over a gap of n second . The series might have periodic similarity/dis-similarity within themselves. How to calculate correlation among the series value in real time?

  • $\begingroup$ autocorrelation or cross-correlation? You need correlation of each series by itself or correlation between 2 series? $\endgroup$ – Kasra Manshaei Oct 26 '18 at 14:21
  • $\begingroup$ correlation between the 2 series $\endgroup$ – Rai Bose Oct 26 '18 at 14:22
  • $\begingroup$ In my answer we assume you are not interested in total similarity $\endgroup$ – Kasra Manshaei Oct 26 '18 at 14:37

well, This is an interesting question! Let's formulate it again to be precise in answer:

I have two real-time streams of values. There are temporary similarities (transient similarities) between two. How to capture it?

As you are talking about similarity, I did not limit the scope to correlation. One may define another measure of similarity. I would say you should define your similarity function first (you may choose correlation. I do as well for my example) and a period of inspection (it's simply a window of length $m$ ending at current timestamp).

Using these two and starting from $m+1$th sample, you can calculate your similarity measure (here correlation) as a time-series. This time-series starts after $m$th sample of input series so keep it in mind to keep similarity measure and original timestamps synchronized. The interval between two consecutive points in output (similarity series) is defined based on the resolution you need. after $m+1$th sample you can shift your window on step at time and calculate the similarity accordingly. Then the intervals in output will be as same as input.

Of course it keeps you more away from real-time as the frequency and number of calculations is maximum. You can define a margine for yourself (let's say $d$ sample) and calculate similarity every $d$ time. Then you lose some info (discarding the samples in between) but you get closer to real-time (you will have $d*n$ seconds to perform computation).

Please let me know if it need clarification. Welcome to the community BTW :)

| improve this answer | |
  • $\begingroup$ thanks alot..can this problem be solved without time-bounding ? $\endgroup$ – Rai Bose Oct 26 '18 at 19:07
  • $\begingroup$ Welcome my friend. Tell me what you mean by tim-bounding and I will answer :) $\endgroup$ – Kasra Manshaei Oct 27 '18 at 1:50
  • $\begingroup$ can i send u a private message somewhere else..maybe ur email? where I can describe the problem in a bit more elaborate way..if you are interested $\endgroup$ – Rai Bose Oct 27 '18 at 6:10
  • $\begingroup$ You can simply google my name and fine me on Linkedin. BTW, if it answered your question you may consider accepting and if you liked it you may consider upvoting. This is the only appreciation as we use our private time for answering questions here :) $\endgroup$ – Kasra Manshaei Oct 27 '18 at 15:16
  • $\begingroup$ i have accepted your ans as the correct ans..however I lack enough points to upvote your ans..i would have gladly done so if I had enough points..thanx $\endgroup$ – Rai Bose Oct 27 '18 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.