I need to do a project using Bayesian statistics for a class and I am trying to apply it to my work.
I help manage a time series database with 40,000+ different time series that we collect. The time series have various frequencies and some of them have frequencies that change over time (was sampled hourly now every 15 minutes for example). Other time series might be seasonal gauge data, only on a few months out of the year. Lastly, some time series that have hourly frequency but only come in once a day or every 6 hours.
We have a few hundred time series that are considered critical that we are warned if the latency (current time - last timestamp) is greater than a specified time. This leaves the vast majority of time series that we are not tracking and I want some way of automatically doing that.
I want to track the latency using conjugate priors. I think I could model it as a gamma likelihood and gamma prior then estimate the posterior every hour. I would then use this to see if the latency falls out of a certain credible interval.
I see a few pitfalls and that this is more or less a time series as well so will it suffer from autocorrelation? Can I do something like take the first difference and proceed?
I have been trying to figure out a good way to track this data but it has been difficult because it is time-consuming to compute. Bayes offers an on-line learning that would be fast and efficient, I would also probably be able to do it all in pure SQL. I am not certain that my approach is valid, however.
Thanks for any input.