1
$\begingroup$

This is a use case that I have and I am trying to automate this. Any pointers would be helpful.

Use Case:

When we deploy any new version of a web service, we keep monitoring it (while deploying to live) to make sure it is not introducing any new errors. For this, we just visually compare with last weeks trend of errors (within the same time frame) and if they look similar, we approve the new build, or if the number of errors seems to increase, we decide to rollback.

What I am looking for is to automate this decision making. Basically, compare the error data during the push with last weeks (or any other time frame) and decide if the two error trends are similar and to what degree they are similar.

The data that I have is x axis -> Time stamp y axis -> # of errors at that time stamp.

I also have details such as number of requests at that timestamp, latency, etc

$\endgroup$
1
$\begingroup$

Your use-case looks solvable using dynamic time warping. DTW is an algorithm for comparing similarity between two temporal sequences.

Implementation of DTW are available in standard data-science programming language via libraries like this R implementation.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Something easy and simple is to get the absolute difference of each point from the two time-series and then use those values to decide.

For example, you can get the average, standard deviation, maximum and minimum of the absolute differences and see in what limits you can safely accept the deployment.

Another way is to use a Granger Causality Test, which is used (among other things) to find if two time-series are statistically different.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.