Determining if time series follows a pattern

Question

I was wondering if anyone had any idea how to solve this problem.

So basically I have a dataset where some person approximately comes at some regular interval and I don't know what that interval is. I need to determine if the person comes in at approximately regular intervals, not necessarily what is the specifically value of the interval. For example if a certain person comes to my house to deliver milk over some period on say

Week 1: Mon, Wed, Fri,

Week 2:Mon, Wed, Fri

Week 3:Mon, Thu, Fri

Week 4:Mon, Wed, Fri

Week 5:Mon, Wed, Fri

Week 6:Mon, Wed, Fri

Week 7:Mon, Wed, Fri

Week 8:Mon, Thu, Fri

So as we can see out of these 8 weeks only in 2 weeks the person didn't come on Wednesday and instead came on Thursday which can be attributed to maybe a holiday the day before. So the solution to this example is that the person does follow a regular pattern.

Similarly this is another example. Say the person came on -

Mon, Thu, Sun, Wed, Sat, Tue, Fri, Mon, Wed, Sun

where this person follows a regular pattern because except for the last Wednesday he comes every fourth day.

This is an example of where a person doesn't follow a pattern, say the person came on

Mon, Wed, Sat, Fri, Thu, Fri, Wed, Fri, Sun, Sun, Sat

I have to do this for hundreds of different people.

Another equivalent problem is if I know which days over a certain period of time (say a month) some person arrives I need to determine if they follow some pattern or not

I thought about trying to fit the data into a sinusoidal curve but I'm not sure if it will work when I have a person who comes every say Mon, Tue, Fri or the 7th of every month or the first and third monday of every month etc.

I'm open to any method as long as it has good accuracy. Also, depending on whichever algorithm you think is best, if possible could you share a link to some code which solves a similar problem so I can get a general idea of how I'm supposed to implement my algorithm. I'm pretty new to machine learning / data science. Thank You!

Answer 1

First of all, digitalize your data on a regular time scale with a time delta shorter than the shortest assumed period. For instance, in the case of the milkman, you could create a table with one record per day having the value 1 if the man come and 0 if not. The table would look like a regular time-serie :

day, presence
  1, 1 
  2, 0
  3, 1
  4, 0

Then perform a autocorrelation analysis of your time-serie. This is a mathematical tool often used in signal processing that consists in, as suggested by @MichelGrogan in his comment, calculating the correlation between the original serie and copies of the serie delayed by distinct time deltas (1 day, 2 days, ...). The result can be plotted as values of correlation in function of the lag value and is called an autocorrelation spectrum. Read autocorrelation for more details.

The max values of the spectrum will give you the values of the most important frequencies observed in your data and confirm that pattern exists. If you do not see clear max values, you can conclude there are no pattern in your data.

Here is an example in Python from your data :

import numpy as np
import matplotlib.pyplot as plt

day =['mon','tue','wed','thi','fri','sat','sun', 'mon','tue','wed','thi','fri','sat','sun', 'mon','tue','wed','thi','fri','sat','sun' ]
num =[    1,    2,    3,    4,    5,    6,    7,     8,    9,   10,   11,   12,   13,   14,    15,   16,   17,  18,    19,   20,   21]
pres=[    1.,    0.,   1.,   0.,   1.,   0.,  0.,    1.,   0.,   1.,   0.,    1.,  0.,    0.,   1.,    0.,  0.,    1.,   1.,    0.,   0.]

corr=np.correlate(pres,pres,'full')
lag=[i-(len(corr)/2) for i in range(len(corr))] 
plt.plot(lag,corr/np.max(corr))

And the spectrum plot :

As you can see there are clear maximum at 2 days and 7 days lags.

Answer 2

It appears as though you are attempting to determine the presence of seasonality in this time series, i.e. a process that repeats itself at regular intervals.

In this regard, it might be a good idea to decompose your time series and check for the presence of seasonality trends.

As an example, this graph in R shows clear seasonality patterns for weather across different seasons:

By decomposing the series, this might give visual cues as to whether seasonality exists among visits by individuals at particular times.