I have a trading system where the model receives 9 time-series and predict :
A - strong down B - week down C - neutral D - week up E - strong up
(these classes are generated from an histogram to have a balanced training dataset ... the histogram is separated in 20% parts of examples centered in 0... accuracy of 20% is the baseline)
For each class I activate a different parametrized trading mechanism...
My model is giving acceptable results. Here is the resulting confusion matrix for val data (28.99% acc):
[[32 20 3 6 8] [35 19 9 7 16] [30 9 6 14 24] [21 14 9 18 29] [ 9 14 3 14 45]]
My question starts here :
I.e. If the model predicts B-"week down" but in reality is A-"strong down" is a miss, but in reality, it will make money...
So in this confusion matrix we can see that it happens 20 times (cell[0,1]) ... also if it is B but the model says A it will make money in 35 trades (cell[1,0])...
And also the same for the UP cases..
All together (from the confusion matrix) : 32+20+35+19 + 18+29+14+45 = 212 winning trades 21+14+9+14 + 6+8+7+16 = 95 losing trades
Assuming negative trades cancel in equal (in reality will not be equal..) positive trades, the total is = 117 winning trades.
What I want is to create a loss function based on categorical_crossentropy but somehow consider:
- pred A real B - half miss
- pred B real A - half miss
- pred D real E - half miss
- pred E real D - half miss
Do not penalize too much this cases. I think this will increase a bit the total number of positive trades. It will guide the learning a litle bit better (maybe not for accuracy but for a better loss that generates a better confusion matrix for profit )...
I have created a custom loss function that reduces 3% the loss for these cases:
def my_loss(y_pred, y_true): y_pre_indexes = K.argmax(y_pred, axis=1) y_test_indexes= K.argmax(y_true, axis=1) TN = K.tf.logical_or( K.tf.logical_or (K.tf.logical_and(K.equal(y_pre_indexes,0),K.equal(y_test_indexes,1)), K.tf.logical_and(K.equal(y_pre_indexes,1),K.equal(y_test_indexes,0))) , K.tf.logical_or (K.tf.logical_and(K.equal(y_pre_indexes,3),K.equal(y_test_indexes,4)), K.tf.logical_and(K.equal(y_pre_indexes,4),K.equal(y_test_indexes,3)))) pos_neg = K.cast(TN, K.floatx()) *(-0.03) + 1 return K.categorical_crossentropy(y_pred, y_true)*pos_neg
(in the code the classes are : 0-A 1-B 3-D 4-E. 2-C is predicting neutral - ignore..)
but fixing to a fixed number of 3% to reduce loss for these cases seems a little bit hard coded.... Something better inside the categorical_crossentropy math philosophy should be better.
Thanks in advance!