5
$\begingroup$

I am really stumped at the moment about how to solve a particular problem. I have many time series like this:

enter image description here

This represents the number of hours a person spends on a website each day throughout the year. Any days where they are not seen to be using the website have zero values, rather than missing values.

What I really want to do is to calculate a metric telling me to what extent there is a consistent "1 hour per day, once per week" pattern, or, say a "10 minutes a day 6 times per week" pattern and I have written some code that produces this pattern. I don't have any background really in machine learning or deep learning, but I have looked into some techniques from spectral analysis and time series. I also found a technique called "Dynamic Time Warping" which I could in theory use to find out how much of a cost there is in mapping this time series onto the "ideal" one (representing the usage patterns I just mentioned).

I tried plotting the periodogram and correlogram, but there seems to be very little correlation indicating periodicity of any kind:

enter image description here

Can anyone shed some light onto how I could use certain techniques to get the information I need, or any introductory texts on periodicity detection or data mining for those who have never really used it before?

Improve this question
$\endgroup$
2
  • $\begingroup$ This sounds like a problem where a matrix profile will be helpful. $\endgroup$
    – Dave
    Commented Dec 3, 2019 at 21:02
  • $\begingroup$ I am afraid that you already did what any introduction to periodicity detection in time series will adivse : plotting the spectrogram and the correlogram. they seems to indicate that you don't have such correlation. Without your data it will be very difficult for us to say if there is any problem with your approach or if the pattern you are looking for isn't just here. $\endgroup$
    – Lucas Morin
    Commented Jan 5, 2020 at 10:46

2 Answers 2

Reset to default
1
$\begingroup$

Time warping is used to align different time series, is more useful for data preprocessing rather than for what you want.

"What I really want to do is to calculate a metric telling me to what extent there is a consistent '1 hour per day, once per week' pattern"

I don't know if you have a specific reason to use a 'metric', if what you need if to find periodical patterns across people what you could do is apply different moving averages to every time series to extract different periodical patterns, then you could evaluate the cross-correlation between different people time series to evaluate which patterns are subjective and which can be instead considered true across people.

Improve this answer
$\endgroup$
1
$\begingroup$

Read up Fourier Transforms - and view the Frequency Spectrum of your data. Also - try manufacturing synthetic data - embedded with patterns you expect to find in the real world. Run this synthetic dataset through the spectrogram and correlogram - That will help gain a visual intuition to differentiate between Signal and Noise.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.