2
$\begingroup$

I want to find outliers in power consumption in real-time, at hourly rate, i.e., at the end of the hour, I should say whether power consumption in current hour was outlier/anomalous or not.

Approach: Till now, I am done with following steps

  1. Say I want to find whether power usage between 9 AM to 10 AM was anomalous? For this, I first find the usage of past n days during the same time interval, then I find the mean/median of all the previous usages
  2. Now, I have usage of the current day and the mean/median usage of previous n days. Which statistical measure should I use to declare whether current day usage was anomalous or not?

Using above approach, for 24 hours of a specific (test) day and using past 10 days consumption, I have obtained results as:

Figure interpretation: Black line represents usage of current hour of current day; red and blue lines represent mean and median of past 10 days for same time interval enter image description here enter image description here

From the visual inspection, I can say that the usage between 07:10 - 08:00 and between 22:10 - 23:00 is anomalous as there is big difference between actual and previous mean/median usage. I don't know which statistical measure should I use to point out such anomalous instances automatically, using the discussed approach.

$\endgroup$
  • $\begingroup$ Look up seasonality correction and change detection in time series. There are plenty of books on this matter. $\endgroup$ – Has QUIT--Anony-Mousse May 28 '16 at 21:22
  • $\begingroup$ I'd say all those are anomalous except 10:30-12:00 and 21:00-22:00? $\endgroup$ – K3---rnc May 29 '16 at 15:51
  • $\begingroup$ You have included the mean and median as reference values, but not the standard deviation. Yet standard deviation is a key component of outlier detection. Are you able to obtain the standard deviation or not? $\endgroup$ – AN6U5 Jun 2 '16 at 23:43
1
$\begingroup$

I have been using one heuristic to detect outlier values that is very simple. Start by calculating an interval centered on the mean and with three standard deviations in radius. Next, calculate a second interval using Tukey's box and whiskers measures, placing the interval limits at the whiskers' limits. Finally, calculate the union of both intervals and use this new interval to detect your outliers: any observation outside this interval is a potential outlier. Please note that you may have to tweak both intervals to calibrate for your desired precision. Also, you may have to take into account possible trends and seasonalities in these intervals, depending on how the historic data behaves.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.