0
$\begingroup$

So, I am trying to use a LSTM model to forecast temperature data on PyTorch. I am relatively new to both PyTorch and the use of recurrent networks so I took a model I found on the internet to start. Unfortunately, I am working with missing data and I am assigning the value 0 to it. The whole project is on github if you need more details.

From the templates, I split it into two approaches.

One train2.py takes a tuple $x=(x_0,...,x_{n-1})$ as input and uses $y=(x_1,...,x_n)$ as target. From there, I recursively call the model $N$ times to forecast $N$ times into the future. Here's the part of the code that matters:

class LSTM(nn.Module):
    def __init__(self, input_size=1, hidden_layer_size=5, output_size=1):
        super().__init__()
        self.input_size = input_size
        
        self.hidden_layer_size = hidden_layer_size

        self.lstm = nn.LSTM(input_size, hidden_layer_size)
        
        self.lstm2 = nn.LSTM(hidden_layer_size, input_size)

        self.hidden_cell = (torch.zeros(1,1,self.hidden_layer_size).double(),
                            torch.zeros(1,1,self.hidden_layer_size).double())
        
        self.hidden_cell_2 = (torch.zeros(1,1,self.input_size).double(),
                            torch.zeros(1,1,self.input_size).double())

    def forward(self, input_seq):
        inpt = input_seq.view(len(input_seq) ,1, -1).double()
        lstm_out, self.hidden_cell = self.lstm(inpt, self.hidden_cell)
        lstm_out2, self.hidden_cell_2 = self.lstm2(lstm_out,self.hidden_cell_2)
        predictions = lstm_out2.view(len(input_seq), -1)
        return predictions.view(-1)

Here's how the output looks like (predict2.html) train2

It kind of gets the oscillatory behavior but the amplitude is way off.

The second one, train.py takes a tuple $x=(x_0,...,x_{\frac{n}{2}-1})$ as input and $y=(x_{\frac{n}{2}},...,x_n)$ as output. For predictions in this one, I make a single call to the model and I can only look at $N<\frac{n}{2}$ points into the future. Here's the code:

class LSTM(nn.Module):
    def __init__(self, input_size=1, hidden_layer_size=5, output_size=1, window=10):
        super().__init__()
        self.window = window
        
        self.hidden_layer_size = hidden_layer_size

        self.lstm = nn.LSTM(input_size, hidden_layer_size)

        self.linear = nn.Linear(hidden_layer_size, output_size)

        self.hidden_cell = (torch.zeros(1,1,self.hidden_layer_size).double(),
                            torch.zeros(1,1,self.hidden_layer_size).double())
        self.sigmoid = nn.Sigmoid()

    def forward(self, input_seq):
        inpt = input_seq.view(len(input_seq) ,1, -1).double()
        lstm_out, self.hidden_cell = self.lstm(inpt, self.hidden_cell)
        predictions = self.linear(lstm_out.view(len(input_seq), -1))
        return self.sigmoid(predictions[-self.window:].view(-1))

The output looks like this:

enter image description here

It fails to keep continuity most of the time, probably because it doesn't have a reference starting point in the output?

With all of that said my question is, how to improve the prediction? Should I add more data to it? Am I using too many hidden layers? Is the use of sigmoid flattening the fluctuations in the data too much? Is the model just completely wrong? I need guidance here.

Thank you for your time.

$\endgroup$

1 Answer 1

0
$\begingroup$

I think I was able to get it working now. What I did was to change the assigned nan value to 25 instead of 0 (which would be about the average) and normalize the values within the interval (-1,1) with a scaled sigmoid shifted on 25.

This is how it looks now:

enter image description here

$\endgroup$
2
  • $\begingroup$ @lcaro, it looks like the model works pretty well. Would you like to clarify what does nan value mean here? Are them the missing values? If possible, would you mind sharing the data set you used? $\endgroup$
    – user288609
    Jun 20, 2021 at 6:11
  • $\begingroup$ @user288609, the NaN values are when there is missing data. I made this telegram bot to visualize and publish information: link. You can download the entire dataset with /get_bmp_data $\endgroup$ Jun 21, 2021 at 11:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.