1
$\begingroup$

I'm trying to forecast Google's stock prices. I've made two models one with LSTM and another one that's Bidirectional LSTM, but the forecasted values don't converge quite well with the test values. I've tried different parameters, but I've hardly got any improvement.

First I had to install these libraries:
!pip install yfinance
!pip install yahoofinancials

Then I import the needed libraries:
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import yfinance as yf
from yahoofinancials import YahooFinancials
import datetime
from datetime import date
from datetime import timedelta  

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Input, Dense, GRU, LSTM, Embedding, SimpleRNN, Activation, Dropout
from tensorflow.keras.optimizers import RMSprop
from tensorflow import keras
from sklearn.metrics import r2_score

Next I set the parameters for the data I'm about to download:
start_date = date(2004, 8, 1)
end_date =  date.today() - datetime.timedelta(days=1)

I download the data:
In [5]: df = yf.download('GOOG', start = start_date, end = end_date, interval = '1d')
Out[5]: [*********************100%***********************]  1 of 1 completed

I visualize the df:
In [6]: df.head()
Out[6]:     Open    High    Low Close   Adj Close   Volume
Date                        
2004-08-19  49.813290   51.835709   47.800831   49.982655   49.982655   44871361
2004-08-20  50.316402   54.336334   50.062355   53.952770   53.952770   22942874
2004-08-23  55.168217   56.528118   54.321388   54.495735   54.495735   18342897
2004-08-24  55.412300   55.591629   51.591621   52.239197   52.239197   15319808
2004-08-25  52.284027   53.798351   51.746044   52.802086   52.802086   9232276

Then I start to transform the data to use it:
In [7]: uni_data = np.array(df['Adj Close'])[1:].reshape(1,-1)
In [7]: uni_data
Out[7]: array([[  53.95277023,   54.49573517,   52.23919678, ..., 2143.87988281,
        2207.81005859, 2132.7199707 ]])

In [7]: data_transpose = uni_data.T

I proceed to add regularisation to the data:
In [8]: tset = (uni_data.T-np.mean(uni_data.T))/np.std(uni_data.T)

In [9]: data_mean = np.mean(uni_data)
In [9]: data_std = np.std(uni_data)

In [10]: tset = tset.T

I generate the batch process:
In [11]: n_steps = 15
In [11]: predict_ahead = 1

In [12]: def generate_batches(uni_data , n_steps, predict_ahead):
    _ , lenght = uni_data.shape
    batchs = lenght - horizonte - predict_ahead + 1
    X = np.zeros((batchs, n_steps + predict_ahead ))
    print('Initial shape of the data : ' , uni_data.shape)
    print('Final shape of the data : ' , X.shape)
    for el in range(batchs):
        data_slice = uni_data[0, el : el + n_steps + predict_ahead]
        X[el , :] = data_slice
    return X , batchs

In [13]: X , batchs = generate_batches(tset , n_steps, predict_ahead)
Out[13]: Initial shape of the data :  (1, 4488)
Out[13]: Final shape of the data :  (4473, 16)

The Train/test split is made:
In [14]: tf.random.set_seed(42)
In [14]: porcentaje_train = 0.8
In [14]: train_range = int(round(len(df)) * porcentaje_train)
In [14]: train = range(train_range)
In [14]: valid = set(range(batchs)) - set(train)
In [14]: valid = np.array(list(valid))

In [15]: X_train = X[train , :- predict_ahead ]
In [15]: X_valid = X[valid , :- predict_ahead ]
In [15]: Y_train = X[train , - predict_ahead :]
In [15]: Y_valid = X[valid , - predict_ahead :]

In [16]: print('X Train shape : ' , X_train.shape)
In [16]: print('Y Train shape : ' , Y_train.shape)
In [16]: print('X Valid shape : ' , X_valid.shape)
In [16]: print('Y Valid shape : ' , Y_valid.shape)
Out [16]: Train shape :  (3591, 15)
Out [16]: Y Train shape :  (3591, 1)
Out [16]: X Valid shape :  (882, 15)
Out [16]: Y Valid shape :  (882, 1)

Then I make the LSTM model

In [17]: epochs = 25

In [18]: def lstm_fit(X_train, Y_train , X_valid , Y_valid, predict_ahead, num_hidden_layers , activ, num_units, loss_type):
    _ , _ , num_vars = X_train.shape
    model1 = lstm(predict_ahead, num_hidden_layers , activ, num_units, loss_type, num_vars)
    hist = model1.fit(x = X_train , y = Y_train, epochs = 20 ,validation_data=(X_valid , Y_valid) , verbose=0)
    return model1

In [19]: tf.random.set_seed(42)

def lstm(predict_ahead, num_hidden_layers , activ, num_units, loss_type, num_vars):
    
    model1 = keras.models.Sequential()
    model1.add(keras.layers.InputLayer(input_shape = (n_steps,num_vars))) 
    for el in range(num_hidden_layers):     
        model1.add(keras.layers.LSTM(activation = activ, units = num_units , return_sequences = True))
    model1.add(keras.layers.LSTM(units = num_units))
    model1.add(keras.layers.Dense(predict_ahead))
    
    model1.compile(optimizer='adam', loss=loss_type, metrics=['mae' , 'mse', 'mape'])
    
    return model1

In [20]: num_hidden_layers = 3
activ = keras.activations.relu
loss_type = keras.losses.mean_squared_error
num_units = 64
dropout_rate = 0.3

Then I make the Bidirectional LSTM model:

First, I made a callback function since the model seemed to be overfitting.

In [21]: class myCallback(tf.keras.callbacks.Callback):
  def on_epoch_end(self, epoch, logs={}):
    if(logs.get('loss')) <0.4:
      print("\nReached below 0.4 loss so cancelling training!")
      self.model.stop_training = True

In [21]: callbacks = myCallback()

In [22]: def bidir_fit(X_train, Y_train , X_valid , Y_valid, predict_ahead, num_hidden_layers , activ, num_units, loss_type, callbacks):
    _ , _ , num_vars = X_train.shape
    model2 = bidir(predict_ahead, num_hidden_layers , activ, num_units, loss_type, num_vars)
    hist = model2.fit(x = X_train , y = Y_train, epochs = epochs ,validation_data=(X_valid , Y_valid) , verbose=0 , callbacks=[callbacks])
    return model2

In [23]: tf.random.set_seed(42)

def bidir(predict_ahead, num_hidden_layers , activ, num_units_bidir, loss_type, num_vars):

    model2 = tf.keras.models.Sequential()
    model2.add(tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)
                                            , input_shape=X_train.shape[-2:]
                                           ))
    model2.add(tf.keras.layers.Dense(32, activation='relu'))
    model2.add(Dropout(0.2))
    model2.add(tf.keras.layers.Dense(6, activation='relu'))
    model2.add(tf.keras.layers.Dense(1, activation='relu'))

            

    model2.compile(optimizer = tf.keras.optimizers.Adam()
                  , loss='mse',metrics=['mae' , 'mse', 'mape'])
    
    return model2

Then I began training the models.

In [24]: X_train = np.expand_dims(X_train ,2)
In [24]: X_valid = np.expand_dims(X_valid ,2)
In [24]: Y_train = np.expand_dims(Y_train ,2)
In [24]: Y_valid = np.expand_dims(Y_valid ,2)

In [25]: X_train.shape
Out [25]: (3591, 15, 1)

In [26]: lstm = lstm_fit(X_train, Y_train , X_valid , Y_valid , predict_ahead, num_hidden_layers , activ, num_units, loss_type)
In [26]: bidir = bidir_fit(X_train, Y_train , X_valid , Y_valid , predict_ahead, num_hidden_layers , activ, num_units, loss_type, callbacks)
Out [26]: Reached below 0.4 loss so cancelling training!

Then I began evaluating and plotting the results.

In [27]: print(lstm.evaluate(X_train, Y_train))
In [27]: print(bidir.evaluate(X_train, Y_train))
Out [27]: 13/113 [==============================] - 1s 8ms/step - loss: 0.0011 - mae: 0.0215 - mse: 0.0011 - mape: 23.0664
[0.0011325260857120156, 0.02146766521036625, 0.0011325260857120156, 23.0664119720459]
Out [27]: 113/113 [==============================] - 0s 2ms/step - loss: 0.3158 - mae: 0.4743 - mse: 0.3158 - mape: 85.1597
[0.31578168272972107, 0.47427594661712646, 0.31578168272972107, 85.15969848632812]

The LSTM returned a MAPE of 23.0664 whereas the Bidirectional LSTM returned a MAPE of 85.1597, which is quite high. This is a problem I found, because no matter how many changes I made to that model, its MAPE was always much higher than I wanted.

In [28]: plt.figure(figsize=(10,10))
In [28]: plt.plot(data_mean + data_std * Y_valid.reshape(-1,1), label = 'Real')
In [28]: plt.plot(data_mean + data_std * lstm(X_valid),  label = 'LSTM')
In [28]: plt.plot(data_mean + data_std * bidir(X_valid),  label = 'Bidireccional')
In [28]: plt.title('Valores Reales y Previstos') #This means Real vs Forecasted
In [28]: plt.grid()
In [28]: plt.legend()
In [28]: plt.show()

The plot returned this: models

As you can see, the models don't fit quite well, at least not during the first half of it, and I've already made a few changes in both models, but the first half never quite fits well.

In [29]: historylstm = lstm.fit(X_train,
                        Y_train,
                        validation_data = (X_valid, Y_valid), 
                        verbose = 0)

In [29]: historybidir = bidir.fit(X_train,
                        Y_train,
                        validation_data = (X_valid, Y_valid), 
                        verbose = 0)

In [30]: lstm_error = pd.DataFrame.from_dict(historylstm.history).iloc[0:1, 7:8].to_string(index=False, header=False)
In [30]: bidir_error = pd.DataFrame.from_dict(historybidir.history).iloc[0:1, 7:8].to_string(index=False, header=False)

In [31]: print(f"LSTM Model - Val MAPE:{lstm_error}")
In [31]: print(f"Bidirectional Model - Val MAPE:{bidir_error }")
Out [31]: LSTM Model - Val MAPE:23.216141
Out [31]: Bidirectional Model - Val MAPE:14.745646

And with this last bit I'm in my current conundrum. For the LSTM model, the training MAPE was 23.0664, and the MAPE for validation is 23.21641 which seems quite a good fit, but for the Bidirectional model the training MAPE was 85.1597 whereas the validation on was 14.745646 which isn't a good fit, neither does it mean it's overfittiing or underfitting it's just quite odd. You would think the low validation results were because of the callback function and the dropout, but the validation MAPE was even lower before I applied those!

Long story short, I have two main issues:
1. The LSTM model would seem a perfect fit when looking at its MAPE, but when its plotted against the real values the fit doesn't seem particularly good. In fact, going by the plot, the Bidirectionl model would seem like the better fit, and still not as good as it should be.
2. The Bidirectional model has an odd issue in which its training MAPE is much higher than the validation one, and yet, when plotted it seems as if doesn't fit that badly.

Basically, I'm at a loss, and I would highly appreciate any help. Thank in advance!

$\endgroup$

1 Answer 1

0
$\begingroup$

LSTM has limits and having the raw data as input might not work in some cases, because it can't learn easily values from ranges that are too far between each other (in your case 3000 is far from 1000). That's why you can do 2 things:

  • Normalize values (ex: minmax), so that LSTMs only learn a range between 0 and 1.
  • Transform your data to relative values (i.e. +X points if the value increase, 0 if stable, -X otherwise).

You should also take into account that LSTMs learn between 250 and 500 values and that it is quite sensitive to noise. There is a lot of noise in stock markets, so you can have even better predictions if you smooth your data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.