1
$\begingroup$

(asked first on stackoverflow but felt like it would be smarter to put it here)

I'm trying to build a Sarima model to predict day by day the expected value of several measures (separately), the point is to detect anomalies when there's more than X values over a certain period that are not inside the confidence interval.
Each days the model is going to retrain over the 2 previous months (for now, that's not fixed until I manage to get better results) and forecast the 24 next hours

Some of those mesures have notable differences on non working days (weekends/holidays or just days where the compagny is closed for other reasons)
Here is an example (may 1 and 2 are weekend days):
Time series visualisation

I've built an array (1 or 0) to represent those non working days and passed it as exog for sarima, something like that :

model = SARIMAX(X_train['value'],exog=train_working_days, order=(p, d, q), seasonal_order=(P, D, Q, 24)).fit(maxiter=100) 

Thanks to that I can clearly see a shift in the predictions, the forecasted values are lower on off-work days :
TIme series with forecast

(I've simulated the data stream on the historical data and reconstructed the series for visualisation and control, but those are separated 24h forecast with a 2 months sliding window for training data)

I know that my forcast is not really precise even on workind days, but most points between the big pikes are really volatile with only a general similar shape, so I'm just using a wide enough confidence interval to test my anomalies with several anomalies needed to reach the alert level (else it would just be noise).

So my issue here is that even if the series are effectively shifted down on off-work days, the daily profile of my forecast remains the same (with degradations) as on working days and as a result the flatter curve lingers on the 1-2 next working days

Is there any way to deal with this kind of things?
I've thought about changing my 24h seasonnality to 7*24h (weekly) and that would probably help for the weekend effect but all others off-work days will still suffer from the same problem.
I've also concidered separating off-works days in a second model, but the information lost would probably be too big for both models to perform better than the unique one.

Maybe I reached the limits of what I can do with Sarima? if so, I would enjoy some suggestions that do not need more than a few months worth of data.

Thank you in advance for reading me, have a nice day

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.