How can I best plot a dataset to visualize the tradeoff between two or more metrics combined?

Question

Basically I have a datasets in which different metrics are present, imagine inference speed and model accuracy.
Some models are fast but have lower accuracy, while some others are some what the opposite, and some are in between.

I want to plot the data so that the combination of such metrics is manifested in the chart and a model that is a bit slower but achieves much higher accuracy is shown to be better than a somewhat faster with lower accuracy.

What kind of a plot does allow me to achieve this? or maybe the correct question is what should I do in such situation?

Answer 1

This can be easily done using a scatterplot where one axis shows one metric and the other shows the other metric. that is, show accuracy as x-axis and inference speed as y-axis. together they can show a tradeoff between both metric.
We can further accentuate the difference based on each point size as well. For example the size of the point signify its higher suitability considering both metric. (simply put, larger points show a high accuracy and high inference)

This can be achieved using a weighted sum of the metrics involved.
To make all of this come together and concrete, here is an example:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# lets create a pandas DataFrame with some dummy data
data = {'model': ['A', 'B', 'C', 'D', 'E'],
        'inference_speed': [1000, 800, 300, 600, 900],
        'accuracy': [70., 80., 90., 85., 65.]}
df = pd.DataFrame(data)

# set the weights for the score calculation
# weight for accuracy
w1 = 0.7  
# weight for inference_speed
w2 = 0.3  

# calculate the score for each model, make sure they have similar scale
df['score'] = w1 * (df['accuracy']/np.max(df['accuracy'])) + w2 * (df['inference_speed']/np.max(df['inference_speed']))

# create a scatter plot with the score as the point size, scale by 10 to make it more pronounced!
plt.scatter(df['inference_speed'], df['accuracy'], s=100*df['score'])

# add a title and axis labels
plt.title('Model Performance')
plt.xlabel('Inference Speed (samples/sec)')
plt.ylabel('Accuracy')

# add annotations for each model
for i, row in df.iterrows():
    plt.annotate(row['model'], (row['inference_speed'], row['accuracy']))

# show the plot
plt.show()

We can get fancy but I guess this addresses the issue just fine.
Note:
If one metric is considerably larger than the other one, we may want to run some kind of transformation on it, like take its log.

Answer 2

I agree with the first answer, this is the most standard option. Here is an alternative, especially if one metric is more important than the other, or if there are more than two metrics:

A simple bar plot where each group of bars represents a model, with a specific color representing a specific metric. The groups/models can be ordered by the most important metric. Disadvantage: the metric must have reasonably similar scales.