16
$\begingroup$

I have been trying to understand how to represent and shape data to make a multidimentional and multivariate time series forecast using Keras (or TensorFlow) but I am still very unclear after reading many blog posts/tutorials/documentation about how to present the data in the correct shape (most examples being of slightly less

My Dataset:

  • several cities
  • for which I have info about say temperature, car traffic, humidity
  • for say the last 2 years (one record for each day)

What I want to do: I'd like to forecast for each city the temperatures I can expect for the next year using a possibly lagged version of temperature, car traffic and humidity (of course there would be several more features but this is just an example for thought).

What I am confused about: If I have 2 cities, for which I recorded 3 features for 365 days. How should I shape my input so that the model can output a forecast for 365 days for these two cities (i.e. 2 time series of temperatures for 365 days)?

Intuitively the tensor shape would be (?, 365, 3) for 365 days and 3 features. But I'm not sure what to stick into the first dimension and, most importantly, I would be surprised if it had to be for the number of cities. But at the same time, I have no idea how to specify into the model that it has to understand the dimensions properly.

Any pointers will be helpful. I'm pretty familiar with the rest of the problem (i.e. how you build a network in Keras etc since I have done this for other neural networks but more specifically how best to encode the sequence for the desired input.)

Oh and also, I guess I could train and predict for each city independently, but I'm sure everyone will agree there are probably things to be learned that are not particular to any city but that can only be seen if considering several of them, hence why I think it is important to encode it in the model.

Improve this question
$\endgroup$
2

1 Answer 1

Reset to default
16
$\begingroup$

The input shape for an LSTM must be (num_samples, num_time_steps, num_features). In your example case, combining both cities as input, num_features will be 2x3=6.

If you lump all your 365 time steps into one sample, then the first dimension will be 1 - one single sample! You can also do sanity check by using the total number of data points. You have 2 cities, each with 365 time-steps and 3 features: 2x365x3= 2190 . This is obviously the same as 1x365x6 (as I said above) - so it would be a possibility (Keras will run) - but it obviously won't learn to generalise at all, only giving it one sample.

Have a look at this relevant question, which I recently answered. There I speak a little about using a rolling window (check the comments of the answer for more info). That will buy you more samples if you need them.

If you want to train a single model with data for both cities as input, then making predictions for both cities at each time-step is as simple as defining a final Dense layer, which outputs 2 units. Your validation/test data must then of course contain a tuple of (city1, city2).

A perhaps more sophisticated way to approach this would be to create datasets on a single-city basis, then train several sub-models on each city individually (say for 5 layers), then Merge/Concatenate them and put several further layers on top. This will mean you are combining the learnt features of each city, which are in turn being combined to a higher level of abstraction. Here is the first image I got from a search engine, which sketches the idea.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Thank you very much for taking the time to answer so clearly! It makes a lot of sense now. But, indeed, what I was fearing is that doing the first solution you explained will prevent from much generalisation. Would the sliding window approach really allow for much generalisation across cities though? $\endgroup$
    – Bastien
    Feb 8, 2018 at 8:34
  • $\begingroup$ You're welcome! Deep nets always benefit from more data. My suggestions above would contain a lot of parameters- with only 365 points, you run the risk of overfitting to those cities (i.e. no generalisation)! You can try data augmentation- how, depends on your data. If you had enough data, you could generalise well - I could imagine a Word2Vec type result (common in text analysis/NLP), where relationships between variables become interpretable. Check this out for info, and this for more intuition/fun. Your results may plot a weather map! $\endgroup$
    – n1k31t4
    Feb 8, 2018 at 9:45
  • $\begingroup$ yes makes sense, so working some rolling windows would kinda trick "more data" in sort of if I understand what you said? $\endgroup$
    – Bastien
    Feb 8, 2018 at 14:31
  • $\begingroup$ Assuming you predict tomorrow based on the last 100 days, the state of the system won't likely have changed as extremely since yesterday (if doing rolling windows), compared to one year ago (not using rolling windows). While it does technically create more samples, you aren't creating new data points - rather predicting allowing the model to focus on smaller chunks one at a time. This higher granularity will allow you to make better (smoother) statistical statements about the results, compared to fitting all the data one, and making one prediction. You decrease uncertainty in results. $\endgroup$
    – n1k31t4
    Feb 8, 2018 at 14:43
  • $\begingroup$ agreed! And would the rolling window not prevent the memory cell from learning long rage dependencies at all? $\endgroup$
    – Bastien
    Feb 8, 2018 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.