1
$\begingroup$

Not sure if this is Math, Stats or Data Science, but I figured I would post it here to get the site used.

As a programmer, when you have a system/component implemented, you might want to allow some performance monitoring. For example to query how often a function call was used, how long it took and so on. So typically you care about count, means/percentile, max/min and similiar statistics. This could be measurements since startup, but also a rolling average or window.

I wonder if there is a good data structure which can be updated efficiently concurrently which can be used as the source for most of those queries. For example having a ringbuffer of rollup-metrics (count, sum, min, max) over increasing periods of time and a background aggregate process triggered regularly.

The focus here (for me) is on in-memory data structures with limited memory consumption. (For other things I would use a RRD type of library).

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I'm not sure I'm understanding what you're looking for, because it seems like a simple log of what and when is all you need. $\endgroup$
    – JenSCDC
    Commented Sep 30, 2014 at 4:13
  • $\begingroup$ @AndyBlankertz in that case I was looking for in-memory, but sure I can push the events to a logfile or redis or similiar. Then I still need datastructure(s) for the efficient calculations. I would like to combine the typical values in a way I dont have to have buckets for all seperate. $\endgroup$
    – eckes
    Commented Sep 30, 2014 at 15:22

1 Answer 1

Reset to default
1
$\begingroup$

It sounds like you would like the Boost Accumulators library:

Boost.Accumulators is both a library for incremental statistical computation as well as an extensible framework for incremental calculation in general. The library deals primarily with the concept of an accumulator, which is a primitive computational entity that accepts data one sample at a time and maintains some internal state. These accumulators may offload some of their computations on other accumulators, on which they depend. Accumulators are grouped within an accumulator set. Boost.Accumulators resolves the inter-dependencies between accumulators in a set and ensures that accumulators are processed in the proper order.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks was looking for ideas for the data structures. Will check out the lib what they use. $\endgroup$
    – eckes
    Commented Sep 30, 2014 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.