0
$\begingroup$

I am currently building a multivariate LSTM for predicting the close price of the next 3 days. I have tried changing parameters such as learning rate, number of layers (and neurons), activation functions, etc. However, I am unable to bring down the validation loss. I have also tried dropout layers and reducing complexity (this hasn't helped improve my results) because I thought it was overfitting. Below is some part of my code

#Getting data
ticker = 'TSLA'
df_stockData = pdr.DataReader(ticker, data_source='yahoo', start='2012-10-20',
                              end=str(date.today() - timedelta(days=1)))
#Scaling
scaler_x = MinMaxScaler(feature_range=(0, 1))
scaler_y = MinMaxScaler(feature_range=(0, 1))
df_stockData_x = scaler_x.fit_transform(df_stockData[selected_features])
df_stockData_y = scaler_y.fit_transform(np.array(df_stockData['Close'].values).reshape(-1, 1))

#PCA
pca = PCA(n_components=0.95)
df_stockData_x = pca.fit_transform(df_stockData_x)
df_stockData_x = MinMaxScaler(feature_range=(0, 1)).fit_transform(df_stockData_x)

#Here I omit some code where I just prep my dataset wherein I use past 21 days data to predict next 3 days
# so input : 21 days data -> output 3 days close price
# Also I use a subset of features, generally 2-3 features (high,low,open etc)


# my model
model = Sequential()
model.add(
    LSTM(512, activation='tanh', recurrent_activation='sigmoid', input_shape=(x_train.shape[1], x_train.shape[2]), 
         return_sequences=True))
model.add(LSTM(256, activation='tanh', recurrent_activation='sigmoid', return_sequences=False))
model.add(Dense(128))
model.add(Dense(64))
model.add(Dense(y_train.shape[1]))
model.add(Reshape((y_train.shape[1], y_train.shape[2]))) 
model.compile(optimizer=Adam(learning_rate=0.001), loss='mse', metrics=[tf.keras.metrics.MeanSquaredError()])
model.summary()

history = model.fit(x_train, y_train, epochs=30, batch_size=5, validation_split=0.3, verbose=1)

Currently, I am getting validation errors around 0.0015- 0.0030. I am hoping to bring it down a little more. Could someone please suggest something ? Apologies if the code is too long. Please let me know how I can make the question better as its my first time.

$\endgroup$
1
  • $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Commented Mar 16, 2022 at 14:46

1 Answer 1

1
$\begingroup$

Stock price prediction is notoriously difficult, especially when trying to predict price. Maybe an approach you could take is convert your prices into returns (helps with the non-stationary nature of raw prices) and then try and model returns? Getting the price from returns is not difficult (multiply current price by predicted return to get expected price.)

Another approach you could take is just more feature engineering, adding further regularisation to the model (dropout, batch-norm) or what has helped me in the past is very small learning rates eg 1e-5.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.