0
$\begingroup$

So, I am trying to use a LSTM model to forecast temperature data on PyTorch. I am relatively new to both PyTorch and the use of recurrent networks so I took a model I found on the internet to start. Unfortunately, I am working with missing data and I am assigning the value 0 to it. The whole project is on github if you need more details.

From the templates, I split it into two approaches.

One train2.py takes a tuple $x=(x_0,...,x_{n-1})$ as input and uses $y=(x_1,...,x_n)$ as target. From there, I recursively call the model $N$ times to forecast $N$ times into the future. Here's the part of the code that matters:

class LSTM(nn.Module):
    def __init__(self, input_size=1, hidden_layer_size=5, output_size=1):
        super().__init__()
        self.input_size = input_size
        
        self.hidden_layer_size = hidden_layer_size

        self.lstm = nn.LSTM(input_size, hidden_layer_size)
        
        self.lstm2 = nn.LSTM(hidden_layer_size, input_size)

        self.hidden_cell = (torch.zeros(1,1,self.hidden_layer_size).double(),
                            torch.zeros(1,1,self.hidden_layer_size).double())
        
        self.hidden_cell_2 = (torch.zeros(1,1,self.input_size).double(),
                            torch.zeros(1,1,self.input_size).double())

    def forward(self, input_seq):
        inpt = input_seq.view(len(input_seq) ,1, -1).double()
        lstm_out, self.hidden_cell = self.lstm(inpt, self.hidden_cell)
        lstm_out2, self.hidden_cell_2 = self.lstm2(lstm_out,self.hidden_cell_2)
        predictions = lstm_out2.view(len(input_seq), -1)
        return predictions.view(-1)

Here's how the output looks like (predict2.html) train2

It kind of gets the oscillatory behavior but the amplitude is way off.

The second one, train.py takes a tuple $x=(x_0,...,x_{\frac{n}{2}-1})$ as input and $y=(x_{\frac{n}{2}},...,x_n)$ as output. For predictions in this one, I make a single call to the model and I can only look at $N<\frac{n}{2}$ points into the future. Here's the code:

class LSTM(nn.Module):
    def __init__(self, input_size=1, hidden_layer_size=5, output_size=1, window=10):
        super().__init__()
        self.window = window
        
        self.hidden_layer_size = hidden_layer_size

        self.lstm = nn.LSTM(input_size, hidden_layer_size)

        self.linear = nn.Linear(hidden_layer_size, output_size)

        self.hidden_cell = (torch.zeros(1,1,self.hidden_layer_size).double(),
                            torch.zeros(1,1,self.hidden_layer_size).double())
        self.sigmoid = nn.Sigmoid()

    def forward(self, input_seq):
        inpt = input_seq.view(len(input_seq) ,1, -1).double()
        lstm_out, self.hidden_cell = self.lstm(inpt, self.hidden_cell)
        predictions = self.linear(lstm_out.view(len(input_seq), -1))
        return self.sigmoid(predictions[-self.window:].view(-1))

The output looks like this:

enter image description here

It fails to keep continuity most of the time, probably because it doesn't have a reference starting point in the output?

With all of that said my question is, how to improve the prediction? Should I add more data to it? Am I using too many hidden layers? Is the use of sigmoid flattening the fluctuations in the data too much? Is the model just completely wrong? I need guidance here.

Thank you for your time.

$\endgroup$
0
$\begingroup$

I think I was able to get it working now. What I did was to change the assigned nan value to 25 instead of 0 (which would be about the average) and normalize the values within the interval (-1,1) with a scaled sigmoid shifted on 25.

This is how it looks now:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.