How to distinguish between normal fluctuation and outliers in ARIMA model?

Question

I have a dataset about sales per day of certain products at the ITEM/DAY/STORE level , I've plotted the series and visually examined it for any outliers, volatility, or irregularities.

And this is what i got :

So there is huge spikes and drops in sales , so i used the tsclean() function - tsclean() identifies and replaces outliers using series smoothing and decomposition-

And this is what i got after plotting :

But now , i'm wondering if what i did was right or not , in the sense maybe those fluctuations were more natural fluctuations than outliers , even though it seems quit weird to me to have a spike in sale of 4000 units in a day and than drops to 2000 the next day.

Any insight would be much appreciated , thank you .

Answer 1

What you have is the fundamental question of statistics, what is noise and what is signal. It’s true that from a modeling perspective it’s easier to model something that’s been nicely smoothed and trimmed, but is that what you want? That’s a hard question to answer in general. I will say that with time data it is quite normal to have seasonal components which I do see in the original but do seem absent in the cleaned data so I err on the side of too clean. I would look for some a more sensitive method than tsclean.