I'm building an FX trading model where I'm trying to predict the +/- movement of a currency pair 5 minutes into the future. I've had some promising results adapting the model as a classifier (i.e., buy if the currency pair is expected to increase by more than some threshold, sell if the pair is expected to decrease by some threshold, or do nothing otherwise), but when trying to create even the simplest multi-layer perceptron regression model, the mean of predicted values bounces around the mean of actual
y values (as expected, this is very close to zero).
The two images below depict the
train predictions (green) versus the actual
y_train values (blue) after two different epochs after a number of epochs had already been completed. Rather than move towards the true
y mean in some stable way, the prediction mean flies right past it and will seemingly do this over and over with more epochs.
So far, I've tried:
- Using a smaller learning rate
- Adding more epochs. The training set has almost 300k items in it, so the classification model is able to learn in only about 5 epochs, but increasing this to 50 or 100 for regression still doesn't seem to help.
- Changing the loss function from
mean squared errorto
mean absolute error
- Changing the optimizer from
Adamto other options like RMSProp (eventually will predict the same value for all items), Adagrad, Adadelta, Nadam, etc.
leaky reluinstead of
relu. Also, if I don't add
yvalues before training, the model ends up predicting the same value for every item after only a few epochs, which might be an extension of the same problem.
- A bias constraint on the output node to keep it close to zero (the true
ymean). Ultimately, it doesn't matter and the model still pushes away from zero.
- Batch normalization
This is happening even with a very simple model such as this:
self.model = models.Sequential() self.model.add(layers.Dense(self.num_layers, activation="relu" input_shape=(self.x_train.shape,))) self.model.add(layers.Dense(1)) self.model.compile(optimizer=self.optimizer, loss='mean_squared_error', metrics=["mae"])
Any suggestions as to how I can stabilize the model?