Timeline for Why is predicted rainfall by LSTM coming negative for some data points?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 5, 2018 at 14:11 | comment | added | kbrose | Did you ever check if you have NaN values in your inputs? Also it's really hard to diagnose those kinds of problems without comprehensive code. Maybe you should ask another question? | |
| Sep 5, 2018 at 7:02 | comment | added | Roy | I am still getting loss:nan after say, about 75% of training time. Tried with ReLU in the middle layer and in last layer as you suggested. Something is wrong somewhere it seems. | |
| Aug 31, 2018 at 5:38 | comment | added | Roy | Thanks for your valuable suggestions. I will try training using ReLU in the last layer as well. Hopefully this time I don't get the loss to go to NA nor the output as negative. | |
| Aug 29, 2018 at 22:28 | comment | added | kbrose | I've updated my answer to try and explain why ReLU needs to be in the last layer. Essentially, what if your Dense layer has a negative weight? You would estimate a negative number. Also, arguments by authority that your data does not contain NA values are less convincing than just asserting that via code. | |
| Aug 29, 2018 at 22:27 | history | edited | kbrose | CC BY-SA 4.0 |
Comment that ReLU needs to be in the last layer
|
| Aug 29, 2018 at 5:53 | comment | added | Roy | I do not have any NA values in my input data series....its official rainfall data for a location collected by an authorised public agency. Regarding ReLU, since I have already have that as my activation function in middle layer, why do I need to add it to the last Dense layer ? | |
| Aug 29, 2018 at 4:11 | comment | added | kbrose | Also you didn’t seem to add ReLU to the layer where it matters (last Dense). | |
| Aug 29, 2018 at 3:58 | comment | added | kbrose | Thank you. Did anything else change between the model without NA and with NA values? Are you sure you have no NA values in your inputs? | |
| Aug 29, 2018 at 3:40 | comment | added | Roy | model = Sequential() #LSTM model model.add(LSTM(128, batch_input_shape=(batch_size, look_back, 1),activation='relu', stateful=True, return_sequences=False)) model.add(Dropout(0.1)) #for better regularization model.add(Dense(1)) model.compile(loss='mean_squared_error', optimizer='adam') | |
| Aug 28, 2018 at 14:03 | comment | added | kbrose | If you share your architecture it will be easier to help. | |
| Aug 28, 2018 at 14:02 | comment | added | kbrose | Are you doing division anywhere in your model? | |
| Aug 28, 2018 at 6:32 | comment | added | Roy | I tried using ReLU but in that case while training, after some epochs, the loss is all of a sudden going to NA. I am not sure why this is happening? | |
| Aug 14, 2018 at 18:44 | comment | added | Roy | Thanks. ReLU should work then, I will try and let you know. | |
| Aug 14, 2018 at 14:19 | history | answered | kbrose | CC BY-SA 4.0 |