I am trying to use tensorflow to predict a decision based on a timeseries dataset.
I have three classes:
The dataset is high imbalanced ~90% of the time the decision is to
"Wait". Thus using
accuracy as a metric is not useful.
I need my model to focus on correctly identifying a pattern that is either
"Backwards", and so I have implemented the following metric to look at Precision and Recall of the classes I deem relevant.
metrics=[tf.keras.metrics.Recall(class_id=1, name='Bkwd_R'),tf.keras.metrics.Recall(class_id=2, name='Fwd_R'),tf.keras.metrics.Precision(class_id=1, name='Bkwd_P'),tf.keras.metrics.Precision(class_id=2, name='Fwd_P')]
On the understanding that they calculate per class.
Precision = TP/TP+FP Recall = TP/TP+FN
I know the formula for F1 but I don't really understand what it is representing, so I am not sure if I should use this?...
F1 Score = 2*(Recall * Precision) / (Recall + Precision)
or should I be using some other type of metric?
For my predicitons, the focus is to correctly identify
"Backwards" amongst the noise of
It would be costly to incorrectly identify
"Fowards" or the other way around, but not so costly to have either identified as
"Wait"s identified as either of the other two.