0
$\begingroup$

I am training a regression model for crypto prediction and the model is not learning.

When I train my model over say the period of 01-01-2021 till 01-01-2022 and I split the dataset into a train and validation dataset (with shuffle enabled) the learning goes very well. Yet this approach performs bad when evaluating other time periods. So now I use different periods for the training and validation data. Since this different approach the model does not learn anymore. The training dataset has 995694 observations. And both the training and testing dataset come from the same function and are equally preprocessed. I have already experimented with less complex models, model parameters, learning rate, normalization & different features.

Does anyone have any ideas what I could try or what I am doing wrong? Or some kind of coding error? Or does anyone have any normalization ideas?

This is the code for the model.

inputs = keras.layers.Input(shape=self.input)
concat = []
for _ in range(4):
    x = keras.layers.Conv1D(64, kernel_size=5, strides=1, dilation_rate=1, padding="same", activation=None, use_bias=False)(inputs)
    x = keras.layers.Conv1D(128, kernel_size=5, strides=1, dilation_rate=1, padding="same", activation=None, use_bias=False)(x)
    x = keras.layers.LSTM(64, activation="tanh", return_sequences=False)(x)
    concat.append(x)
x = keras.layers.Concatenate(axis=1)(concat)
x = keras.layers.Dropout(0.2)(x)
x = keras.layers.Dense(512, activation="swish")(x)
x = keras.layers.Dropout(0.2)(x)
x = keras.layers.Dense(256, activation="swish")(x)
outputs = keras.layers.Dense(self.output, activation="linear")(x)
self.model = keras.models.Model(inputs=inputs, outputs=outputs)
optimizer = 
self.model.compile(
    loss="mae",
    optimizer=keras.optimizers.Adam(learning_rate=0.00001),
    metrics=[precision],
    run_eagerly=False,
)

Training logs.

Epoch 1/250
3890/3890 [==============================] - 53s 12ms/step - loss: 1.1086 - precision: 0.5211 - val_loss: 1.0145 - val_precision: 0.5168
Epoch 2/250
3890/3890 [==============================] - 45s 11ms/step - loss: 1.1059 - precision: 0.5245 - val_loss: 1.0150 - val_precision: 0.5182
Epoch 3/250
3890/3890 [==============================] - 48s 12ms/step - loss: 1.1034 - precision: 0.5270 - val_loss: 1.0130 - val_precision: 0.5213
Epoch 4/250
3890/3890 [==============================] - 129s 33ms/step - loss: 1.1001 - precision: 0.5283 - val_loss: 1.0141 - val_precision: 0.5201
Epoch 5/250
3890/3890 [==============================] - 50s 13ms/step - loss: 1.0972 - precision: 0.5309 - val_loss: 1.0142 - val_precision: 0.5268
Epoch 6/250
3890/3890 [==============================] - 45s 11ms/step - loss: 1.0943 - precision: 0.5309 - val_loss: 1.0153 - val_precision: 0.5235
Epoch 7/250
3890/3890 [==============================] - 45s 12ms/step - loss: 1.0918 - precision: 0.5320 - val_loss: 1.0149 - val_precision: 0.5195
Epoch 8/250
3890/3890 [==============================] - 45s 11ms/step - loss: 1.0888 - precision: 0.5342 - val_loss: 1.0166 - val_precision: 0.5223
Epoch 9/250
3890/3890 [==============================] - 45s 11ms/step - loss: 1.0857 - precision: 0.5351 - val_loss: 1.0208 - val_precision: 0.5233
Epoch 10/250
3890/3890 [==============================] - 43s 11ms/step - loss: 1.0827 - precision: 0.5362 - val_loss: 1.0183 - val_precision: 0.5192
Epoch 11/250
3890/3890 [==============================] - 44s 11ms/step - loss: 1.0805 - precision: 0.5382 - val_loss: 1.0198 - val_precision: 0.5214
Epoch 12/250
3890/3890 [==============================] - 44s 11ms/step - loss: 1.0786 - precision: 0.5399 - val_loss: 1.0191 - val_precision: 0.5230
Epoch 13/250
3890/3890 [==============================] - 45s 12ms/step - loss: 1.0769 - precision: 0.5401 - val_loss: 1.0158 - val_precision: 0.5262
07-02-22 12:34:21 - Saving model weights to /home/paperspace/trader/.cache/weights/linear/neuralnet.1.1.19.h5 ... done
07-02-22 12:34:21 - Trained the model in 11.4m.
Training evaluation: loss: 1.1004 - precision: 0.5312
Validation evaluation: loss: 1.013 - precision: 0.5213

This is a scatter plot comparing the training and validation dataset. Scatter Plot

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

A NN model with standard dense layer usually performs „bad“ on time series. You may try a LSTM model.

However, especially with time series like stock prices etc. you often face a „random walk“, which is hard (or impossible) to predict in a good way by now.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I indeed already tried models with 1, 2 or 3 LSTM layers with and without dropout. It faces the same problem. Where 2 LSTM layers without dropout performed the best. $\endgroup$
    – majomere
    Feb 7, 2022 at 22:40
  • $\begingroup$ So I suspect you face the „random walk“ problem. Crypto currencies follow a random walk. Try a linear model with one lag. If you see a coefficient of the lag = 1, you face a random walk. $\endgroup$
    – Peter
    Feb 7, 2022 at 22:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.