Trying to make compelling plot for classification results with python

I've been working on a classification problem and have some good results, but now I struggle with trying to put together a good plot to illustrate the probabilities for each prediction. Here is my current data:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

prob    actual  pred    correct
0   0.460200    0   0   1
1   0.548478    1   1   1
2   0.270609    0   0   1
3   0.686557    0   1   0
4   0.527935    0   1   0
5   0.098687    0   0   1

I've been able to create a bar chart with probabilites using this code:

plt.bar(np.arange(len(voting_predictions[:,1])), voting_predictions[:,1])
plt.xlabel("record number")
plt.ylabel("probability")
plt.title("Classification Probabilities")
plt.show() However, I'm thinking there has to be a better way to include more information and a key. I'd like the "correct" labels to be clear to see, so from a visual standpoint you can see how the probabilities relate to a correct classification.

What piece of information are you trying to convey by presenting this plot? That determines what an appropriate plot type would be.

The bar chart you show in your question would be useful if the specific indices of the probabilities are important - for example, if there could be something special about e.g. indices 504-559 and a person would want to see what probabilities go with those indices - but I suspect that's not the case. It's more likely that you're interested in how the probabilities correlate with actual results. To show that, you could start with a simple scatter plot with probability on the horizontal axis and actual result on the vertical axis. It also helps if you use different colors (and perhaps slightly different symbols) for actual results of 0 and 1. Here's a primitive example: Another option to show nearly the same information would be to group the probabilities into ranges and plot a histogram showing how many 1 results fall in each range. This better conveys how narrow the crossover from 0 to 1 predictions is. If all you want to convey is how many predictions of each type were correct vs incorrect, then the confusion matrix recommended by StatsSorceress might be a better choice, since it more directly presents that information. You could "decorate" it as a heatmap, if you think that would enhance the impact (i.e. if this is targeted at less numerically-minded people or it fits with an overall graphical theme), but that probably doesn't make much of a difference. It's really rare that you'd show a plot of the probabilities for each example in your set. Are you sure you want to do this? A better presentation might be a confusion matrix. Here's how it works:

1) The columns are the true class labels

2) The rows are the predicted classes

3) Along the right hand side of the plot you can show the probability of correctly assigning to a class (or the classification error, if you prefer).

For example, say I have three classes in my dataset. I have 10 examples of each class, so 30 examples total.

My classification results from my model are:

8 of the 10 examples in class 1 were correctly labelled; 1 was misclassified to class 2 and 1 was misclassified to class 3.

7 of the 10 examples in class 2 were correctly labelled; 3 were misclassified to class 3.

9 of the 10 examples in class 3 were correctly labelled; 1 was misclassified to class 2.

Then my confusion matrix looks like this:

$\begin{bmatrix} 0.8 & 0.1 & 0.1 \\ 0.0 & 0.7 & 0.3 \\ 0.0 & 0.1 & 0.9 \end{bmatrix}$

And along the right hand side of my confusion matrix I can include the classification error: 0.2 for class 1; 0.3 for class 2; 0.1 for class 3.

Notice the rows must add to 1.