0
$\begingroup$

I am not sure I am thinking about my problem the right way, so I am looking for the right approach.

I have a data set that, for the sake of argument, has a mean of 1 and a standard deviation of $\sigma$. It is time series data and I have converted it so that it is stationary and then trained a LSTM model to predict the next value $x$ from the previous $n$ observations where $n$ is relatively small, say 15 observations. When I evaluate the model, I capture the error for each prediction along with MSE and MAE, but the model uses the MAE as a loss function.

My task is to predict $x$ and compare it to an observed value. If the observed value is 'too far' away from the prediction, I want to check it as it may be a bad value. What I am struggling with is how to define 'too far'. Intuitively, I want to define 'too far' as 3 standard deviations of error away from the predicted value of $x$. But this somehow doesn't seem right, or at least, I don't see much discussion of this approach.

My question is, should I compute the standard deviation of the error and define my test using this measure, or should I use the standard deviation of the raw data? If I can use the standard deviation of the error, can/should I calculate it based on the MAE or the raw error values?

Any guidance would be helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.