Timeline for Adding Features To Time Series Model LSTM
Current License: CC BY-SA 3.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 20, 2019 at 13:22 | comment | added | abeboparebop | This is a very nice answer. Might be worth noting that in the RNN "estimator" (i.e. built-in model with all the bells and whistles) in TensorFlow, they appear to concatenate the static contextual features onto each element of the sequence, as in your suggestion #2. I wonder if there is a good reason this is their default approach, rather than the initialization-style approach in suggestion #3. | |
| Oct 25, 2018 at 12:23 | comment | added | roberto tomás | I'd appreciate an example of a LSTM with exogenous data .. like if variables correlate in two groups (like: performance_walking, performance_chewing_gum, hours_awake, amount_of_work_to_do -- the first two correlate, the second two correlate, and the second group influences the first, but the trend lines of the first group will be relatively smooth and upwards, while the trend lines of the second group will be far more noisy). In such cases a naive LSTM might do well with the larger group and poor with the smaller. | |
| Jan 9, 2018 at 16:59 | comment | added | StatsSorceress | For Keras users: if you want to include an external constant (like a conditioning variable) in a recurrent network, see "Note on passing external constants to RNNs" here: keras.io/layers/recurrent | |
| Jan 9, 2018 at 16:52 | comment | added | StatsSorceress | @AdamSypniewski, are you sure that recommendation is what those papers are doing? My read is that Karpathy & Fei-Fei are encoding an image to h-space via one model (a CNN), and encoding a sentence into h-space via a separate model (an LSTM), and then they infer from there; I think Vinyals et al are feeding in their conditioning variable at time '-1' before the start of the sequence. Am I wrong? | |
| S Nov 5, 2017 at 9:29 | history | suggested | user2614596 | CC BY-SA 3.0 |
you wrote to add the input to the hidden state, instead of the projected condition variable. I've added a detail about using the Dense layer because the edit must be at least 6 characters
|
| Nov 4, 2017 at 17:06 | review | Suggested edits | |||
| S Nov 5, 2017 at 9:29 | |||||
| Feb 24, 2017 at 16:16 | comment | added | Adam Sypniewski | Great question! I've included edits to address this. | |
| Feb 24, 2017 at 16:15 | history | edited | Adam Sypniewski | CC BY-SA 3.0 |
Added additional information about conditioning an RNN
|
| Feb 23, 2017 at 18:32 | comment | added | horaceT | OK, here is a very artificial example. Say you're trying to predict weather at time t, based on obs from last n time steps. Weather depends on the part of the world you're in. If it's summer in northern hemisphere, it's winter in southern hemisphere. So this north/south factor should be taken into account. Can you incorp it into LSTM? | |
| Feb 23, 2017 at 18:27 | comment | added | Adam Sypniewski | Could you give an example? | |
| Feb 23, 2017 at 18:26 | comment | added | horaceT | Good suggestion, but what if the output of the LSTM has structural dependence on a non-time series predictor. | |
| Feb 23, 2017 at 13:06 | history | answered | Adam Sypniewski | CC BY-SA 3.0 |