1
$\begingroup$

My dataset is performance metrics (response time) of a web page over the course of a single day. The data looks roughly like this:

| Date | User Id | Response time in milliseconds |
|------+---------+-------------------------------|
| ...  | U1      |                           390 |
| ...  | U2      |                          1965 |
| ...  | U2      |                          7789 |
| ...  | U1      |                           479 |
| ...  | U1      |                          9876 |
  • Charting the percentiles 50th, 75th, 90th, 95th and 99th I could see that the response time for the vast majority of users is below 600 ms - which is seen as acceptable.
  • But there are extreme spikes in response times: I tried to sketch that in the data example above.
  • What I would like to do is visualize if it's always the same users affected by these spike or if the behaviour is more erratic than that. In the example above, user U2 would experience frequent spikes, whereas U1 has better performance with one spike (I know the sample size is obviously small in the example but I hope it helps to illustrate what I'm after.)
  • Goal of this effort is to reveal if it's always the same few users that are affected by spikes, then we could narrow down the problem.

An idea would be to use a bubble chart, which visualizes the following triplet for each user:

  1. Count of how often we saw the user id during the day. (This will be the size of the bubble)
  2. 99th percentile for this user (x-axis)
  3. 75th percentile for this user (y-axis)

What I thought we could see from such kind of chart:

  • The bigger the bubble, the more frequent we saw the user.
  • The closer to 0 on both axes, the better the overall performance.
  • The further away from 0 on the x-axis, the more extreme the performance peaks.
  • The further away from 0 on the y-axis, the worse the common case performance.

Assumption is, users that are shown as large bubbles near the upper right corner could be the ones to investigate.

I thought a while about this, but I can't really say I'm totally convinced this is what I'm after. Does this approach make sense to find out if we have mostly the same users experiencing the worst performance?

$\endgroup$
1
$\begingroup$

One option would be look at conditional probability based on percentiles.

First, find percentiles based on all the data. For example, 99th percentile is 1432 milliseconds.

Then, find percent of a specific user request above that threshold. For example U1 has 50% of requests above 99th percentile.

This could be made into a cross-tabs table for easier exploration. The rows would be each user and the columns would be binned response time percentiles.

$\endgroup$
1
  • $\begingroup$ Thanks for the answer. I will need to try this out. Since we have a few thousand unique users, we'd need to filter such a table - exclude everyone with low probability (like 25% or less) to be in a certain percentile and focus on the user which have the highest probability, maybe .. $\endgroup$
    – Max
    May 2 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.