# Underlying model for prediction using different prediction variables

I have time-series energy consumption data for a duration of one-month. The frequency of data is half-hourly. The features of dataset are

• temperature - temperature value at particular time instant
• humidity - humidity value at particular time instant
• timeofday - this corresponds to 48 half-hourly durations of a day.
• weekday - day of the week (value between 1 - 7)
• prevday1 - energy consumption at the same time-instant on previous day
• prevday2 - energy consumption at the same time-instant on a day before the previous day
• prev_instant1 - energy consumption at the previous time instant. For example, for 7 pm, previous_instant1 is 6:30 pm
• prev_2_hour - average energy consumption during the last two hours

Using these variable I need to forecast power (energy) consumption. The pairs plot of these variables is

I started with a linear model, but it seems that the model is performing very badly. I want to know is there any automatic technique, which can check all possible combinations of predictor variables and output the best fitted model. I can check all models (linear combinations, non-linear, splines etc) manually, but it will take huge time.

## UPDATE

I am forecasting in real-time at a half-hourly rate (very short term forecasting). Also, I use separate models corresponding to different half-hours of the day. For example, In a day we have 48 half-hours so corresponding to each half-hour I have a separate model. For the whole process, I use following approach:

1. Create a training data set with above mentioned features. This training data set is of one month duration (30 days)
2. In the next step, I forecast for the next day (testing day) using specific models for each of 48 half-hour durations
3. At the end of the day, I retrain all my 48 models with recently updated data (recent historical 30 days)

Steps 2 and 3 continue till stopping condition. Stopping condition used is the no. of testing days. A screenshot of my training data is as:

head(training_data)


tail(training_data)


• How far ahead do you want to forecast? Because prevday2 doesn't exist if you are trying to forecast less than two days. You want to forecast a week ahead - you can't use any of the prev variables. That will simplify things a bit. Also its impossible to see what's going on with the power usage variable in your plot - since its strictly positive, why not log-transform it? Once you've sorted all that out then look at model variable selection. Or just throw everything into a Random Forest model... – Spacedman Apr 6 '16 at 7:45
• I have explained my problem in more detail. – Haroon Rashid Apr 6 '16 at 8:11
• What does the power consumption curve look like? How good an estimate of the next half-hour is the current half-hour? What about how good an estimate of the next half hour is the trend based on the last two half hours? Where's your justification for 48 (independent?) models? if you really only want to do one-step ahead forecasting then Kalman filter and job done. – Spacedman Apr 6 '16 at 10:03