4
$\begingroup$

As a part of a statistical analysis engine, I need to figure out a way to identify the presence or absence of trends and seasonality patterns in a given set of time series data. While most answers and tutorials in the Internet outlines methods to predict or forecast time series data using machine learning models, my objective is simply to identify the presence any such pattern.

Example: daily sales data through a year

This data set could show upward trends with monthly seasonality or no actual trend with yearly seasonality.

If I am not to manually inspect the scatter plot of the distribution, in what ways can I determine the presence of those patterns?

I have looked at the following approaches so far:

  1. Using moving average or exponential smoothing to smooth the time series curve and then check if the resulting line can be approximated to a linear curve, which is supposed to provide an upwards or downward trend if any.

  2. Using auto correlation to check for seasonality, which I am yet to confirm if it's a possible solution or not (any ideas?)

Improve this question
$\endgroup$

3 Answers 3

Reset to default
4
$\begingroup$

Another idea could be Fourier Transformation, which takes a time serie as an input (time domain), and converts it into frequency domain.

Consider this example: enter image description here enter image description here

When you transform the time series data from time domain into frequency domain, you can observe the repeated patterns (=seasonality). In this case, there is peak at 12 (day/night rhytm), at 24 (day) or 168 (week).

I can imagine you can perform the FFT and then extract the peaks based on certain threshold

The Git repo also contains a nice introduction into FFT.

Source: https://github.com/balzer82/FFT-Python

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you, this is close to the actual solution i found (forgot to update the answer). The only issue was that the cyclic pattern interferes here with seasonality and sometimes gives incorrect seasonality. Any idea on how this could be adjusted? $\endgroup$
    – ManZzup
    Commented Aug 2, 2018 at 6:27
  • $\begingroup$ If you know the cyclic pattern, you could remove it before you perform the transformation, no? $\endgroup$
    – HonzaB
    Commented Aug 2, 2018 at 7:04
  • $\begingroup$ yes but issue is the cyclic patterns are unknown as well $\endgroup$
    – ManZzup
    Commented Aug 3, 2018 at 17:31
0
$\begingroup$

You could calculate the RSI (relative strength index) of your sales, over 1 month, 3 months, a year or however long of a time interval you wanted to measure if the current positive trend is outweighing the negative trend over that amount of time (or vice versa).

You could also try ADX (average directional movement index) combined with +- directional indicators to again detect the strength of the trend, and also identify if it is a positive or negative one depending on which directional indicator outweighs the other.

I'm sure there are many other traditionally stock oriented statistics you could use too.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thank you for the ideas, I will surely followup on it but it seems my example it bit too specific to sales, i actually need this to be able to detect trends in any other field as well, ex: logistics, HR etc. will these methods still work on them? $\endgroup$
    – ManZzup
    Commented Jul 7, 2017 at 4:09
  • 1
    $\begingroup$ These techniques can be applied to any series of values, just like a moving average. Have a read of the links I included to get an understanding of what they show and how they might help you :) $\endgroup$
    – Dee Carter
    Commented Jul 7, 2017 at 8:23
0
$\begingroup$

Classically, autocorrelation is how you model time-series data and imply seasonality. This is mostly mathematical but not much related to machine learning. Once you have the model, RSI/ADX, or other momentum index, EWMA or other moving average, are smoothing technique which help you to generate signals, which enable you to know where exactly you are in the time series.

If you want to compare the similarity of two time series with the same time index, you may compare their euclidean distances, or compute a covariance matrix. These allow you to explore the data better and decide what you can do next.

Machine Learning algorithm require labelled data for training. You may provide a specific context of your use case to see if there are some mature machine learning algorithm to be applied on your time-series data

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.