0
$\begingroup$

Context:

I have a time series on which I've trained an autoencoder for anomaly detection (AD) purposes.

My time-series is 6 years worth of daily data, but for AD purposes, I am looking at one week window. This means I've only got 330 sub-sequences for my purposes.

I used 10% of those sub-sequences as my validation set, so 33 samples. It's not much, but it my model generalizes well to these few samples.

Problem:

For AD, I want to estimate the parameters of the residual distribution. I need these estimates in order to calculate an anomaly score using Mahalanobis distance. However, I don't think 33 samples is enough to estimate such parameters. Using the entire dataset seems like a bad idea, a sort of data leak.

Question:

Is it possible to synthesize new data in order to expand my validation set?

I thought of adding "random" noise to some samples, but I'm pretty sure that would make the parameter estimates skewed as it might change the distribution in the wrong direction.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.