Complete newbie here so, do forgive any misunderstandings that I may have.
Currently, we have a lot of metrics to track each detail of our backend applications like traffic, response times, memory load, upstream response times, etc using a time-series solution (Prometheus) and have dashboards similar to what MediaWiki has
When an issue happens like a sudden spike in response times, we go over the graphs and try to manually figure out the issue from the graphs. The steps to debug are standard which can be represented easily with a flowchart
I would like to automate these manual steps and as I understand, I need to build a basic understanding of Data Science for things like:
- Understanding whether a spike has occurred
- Comparing the trends between different time-series data to be able to conclude if one is the cause of the other.
- Even if the solution can't find the right match but it would still be a win if it can narrow down the areas for users to look into.
- Ignoring minor fluctuations to understand the overall trend
For this, I have the following questions:
- What are the areas or concepts that I should be familiarizing myself with?
- What existing solutions or building blocks like modules I can use for my use case?
To add, I am not looking for a forecast here and the data will be continuous.