Python - Get FP/TP from Confusion Matrix using a List

Question

I using two different classifiers to predict a binary target (Random Forests and Decision Trees). Now I want to evaluate my model creating a confusion matrix. For example, for predicting the binary value using random forests I've:

training_features, test_features, training_target, test_target, = train_test_split(df.drop(['score_goal'], axis=1),
                                              df['score_goal'],
                                              test_size = .3,
                                              random_state=12)

clf_rf = RandomForestClassifier(n_estimators=25, random_state=12)
clf_rf.fit(training_features, training_target)
print("Accuracy using Random Forest Classifier is ", clf_rf.score(test_features, test_target)*100)

I'm confusing because I don't know how I can compare the predicted values to identify how many False Positives, etc. I have.

Anyone knows how can I build that function?

Thanks!

Answer 1

Looks like you're using scikit-learn. So why not explore a bit more? Scikit has a metrics module, that can be of use for your problem. Essentially, what you need is to have two separate arrays - one with real labels and another with predicted labels. And then you're good to go, you could call metrics.classification_report or metrics.confusion_matrix or metrics.accuracy_score, all of them use the real labels and the predicted labels.

There's nothing wrong with using clf_rf.score(test_features, test_target), but it will only give you a single value. If you look at the source code, what happens is that the score method calls a predict method with test_feature for prediction of labels, which occurs behind the scenes.

It's better to actually capture those predicted labels, so that you can reuse them.

clf_rf.fit(training_features, training_target)
predicted_target = clf_rf.predict(test_features)
accuracy = sklearn.metrics.accuracy_score(test_target, predicted_target)
cnf_matrix = sklearn.metrics.confusion_matrix(test_target, predicted_target)
class_report = sklearn.metrics.classification_report(test_target, predicted_target)

And then you could do whichever you like with the calculated metrics, print, plot, etc. Have a look at the examples that are included for each model/metrics you're using 1, 2, 3, 4, etc.