1
$\begingroup$

I have two files with accelerator readings and I want to get some metric/ measurement to get the similarity between these two files. I have tried Pearson’s R coefficient, dtw distance, dtw score. Pearson’s r gives returns a value 1 if the files are identical, the dtw score and path are 0 if the files are identical.

But I need a solution if the files are as the ones in the figures, similar, with a little time lag. They are readings from two different accelerators who were attached to the same source. The sampling frequency and amplitude is not same. Even the number of readings are not same. Time stamps could be different.

How do I measure the similarity between such files? Is there some metric or measurement I can get using Python? Because dtw score and dtw distance do give some output, but there is no way I can say the files are similar using those values.

File 1

File 2

Improve this question
$\endgroup$
5
  • $\begingroup$ I like to start simple. Why not start with DTW? It is textbook, simple, and effective. $\endgroup$
    – EngrStudent
    Oct 18, 2022 at 14:45
  • $\begingroup$ I did try that. Did not give me desired results.. $\endgroup$
    – Chaitra
    Oct 26, 2022 at 11:57
  • $\begingroup$ Can you share how it worked? I'm asking because folks tend to engage when effort is shown. It is recommended to discuss what you tried, why you tried it, and how it turned out. Was it multivariate dtw? $\endgroup$
    – EngrStudent
    Oct 26, 2022 at 13:14
  • $\begingroup$ It worked only for exactly the same signals, but for signals as above, it gave a numerical answer which was not concrete enough for me to say that they are similar. It was univariate that I tried. I calculated RMS and then tried. $\endgroup$
    – Chaitra
    Oct 27, 2022 at 4:29
  • $\begingroup$ It looks like there are 4 modes in these two plots: nearly flat, thin chunks (first 5 chunks), medium chunk (following the first 5 thin chunks), and fat chunks (2x). I think if you segmented the first one, you could get "templates" and match those in the first one. It would give you window of DTW distance to map each, because the skinny chunks look like each other, but not the medium, large, or zero. You could then compose the time-series as a sequence of 0 (for flat), 1 (for thin), 2(for medium), and 3(for fat values). apply to 2nd series to get SAX distance (or hamming?) $\endgroup$
    – EngrStudent
    Oct 27, 2022 at 14:41

2 Answers 2

Reset to default
0
$\begingroup$

What you could do is

  • downsample both series to match sampling rate between them
  • crop the longest series to match the shorter one

Then, there is a variety of methods to estimate similarity between the two series. Some of those are:

  • cross correlation: this will be affected by the amplitude and will not be able to estimate lagged correlations, prone to noise.
  • coherence: normalised frequency based correlation (cross-spectrum), not prone to amplitude or noise.
  • wavelet coherence: similar to above but based on wavelet transformations instead of STFFT.
  • dynamic time warping: measuring similarity between two temporal sequences, which may vary in speed.
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your response. DTW gives score and output but there is no threshold value that can say that they are highly similar. Are there any libraries you suggest for coherence and wavelet coherence in Python? $\endgroup$
    – Chaitra
    Dec 8, 2020 at 15:09
  • $\begingroup$ Unfortunately, not aware of any substantial libraries for DSP in python. Not sure if familiar, but I would suggest going with MATLAB or Octave if you do not have a license for it. $\endgroup$
    – hH1sG0n3
    Dec 8, 2020 at 15:17
  • $\begingroup$ Thanks. But I will have to search for something in Python. $\endgroup$
    – Chaitra
    Dec 8, 2020 at 15:35
0
$\begingroup$

Maybe you could use Matrix Profile (see Matrix Profile Foundation) to solve your problem.

A Matrix Profile is a new time series that measures the similarity of one time series to another, or a "self similarity" measure of parts of a time series to other parts of the same series. There is a parameter, usually called sub_len, which could be adjusted for you use case, or alternatively you could use what's called a Pan Matrix Profile to investigate different length sub_len in one go.

The main claim by Matrix Profile proponents is that, given a Matrix Profile, other time series tasks then become (trivially?) easy. The above linked page has links to a Github with Python code.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.