I have time-series energy consumption data for a duration of one-month. The frequency of data is half-hourly. The features of dataset are

  • temperature - temperature value at particular time instant
  • humidity - humidity value at particular time instant
  • timeofday - this corresponds to 48 half-hourly durations of a day.
  • weekday - day of the week (value between 1 - 7)
  • prevday1 - energy consumption at the same time-instant on previous day
  • prevday2 - energy consumption at the same time-instant on a day before the previous day
  • prev_instant1 - energy consumption at the previous time instant. For example, for 7 pm, previous_instant1 is 6:30 pm
  • prev_2_hour - average energy consumption during the last two hours

Using these variable I need to forecast power (energy) consumption. The pairs plot of these variables isenter image description here

I started with a linear model, but it seems that the model is performing very badly. I want to know is there any automatic technique, which can check all possible combinations of predictor variables and output the best fitted model. I can check all models (linear combinations, non-linear, splines etc) manually, but it will take huge time.


I am forecasting in real-time at a half-hourly rate (very short term forecasting). Also, I use separate models corresponding to different half-hours of the day. For example, In a day we have 48 half-hours so corresponding to each half-hour I have a separate model. For the whole process, I use following approach:

  1. Create a training data set with above mentioned features. This training data set is of one month duration (30 days)
  2. In the next step, I forecast for the next day (testing day) using specific models for each of 48 half-hour durations
  3. At the end of the day, I retrain all my 48 models with recently updated data (recent historical 30 days)

Steps 2 and 3 continue till stopping condition. Stopping condition used is the no. of testing days. A screenshot of my training data is as:


enter image description here


enter image description here

  • 1
    $\begingroup$ How far ahead do you want to forecast? Because prevday2 doesn't exist if you are trying to forecast less than two days. You want to forecast a week ahead - you can't use any of the prev variables. That will simplify things a bit. Also its impossible to see what's going on with the power usage variable in your plot - since its strictly positive, why not log-transform it? Once you've sorted all that out then look at model variable selection. Or just throw everything into a Random Forest model... $\endgroup$
    – Spacedman
    Commented Apr 6, 2016 at 7:45
  • $\begingroup$ I have explained my problem in more detail. $\endgroup$ Commented Apr 6, 2016 at 8:11
  • $\begingroup$ What does the power consumption curve look like? How good an estimate of the next half-hour is the current half-hour? What about how good an estimate of the next half hour is the trend based on the last two half hours? Where's your justification for 48 (independent?) models? if you really only want to do one-step ahead forecasting then Kalman filter and job done. $\endgroup$
    – Spacedman
    Commented Apr 6, 2016 at 10:03

1 Answer 1


Have had the same problem before. Used Weka tool (http://www.cs.waikato.ac.nz/ml/weka/).

The approach:
• convert time of day into a nominal (non-numeric), eg bins numbered 0-47
• convert day of week into a nominal (non-numeric)
• use M5Tree to construct a model

The rules in the tree will pop out things like "if 0 < bin < 5:30am_bin" then ... which discriminates start of day

It is better with more than 1 month of data. There will only be 4 samples for certain days of the week, for example. So more months would be better. And in such case again converting month to a nominal value (non-numeric) lets the rules pick up seasonal and monthly changes.

  • $\begingroup$ Welcome to SE. Your answer isnt clear if you are answering the original question, or saying you have the same problem. Normally if you don't have a full answer, then putting ideas in as comments can be very helpful. $\endgroup$
    – Marcus D
    Commented May 3, 2016 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.