0
$\begingroup$

I using two different classifiers to predict a binary target (Random Forests and Decision Trees). Now I want to evaluate my model creating a confusion matrix. For example, for predicting the binary value using random forests I've:

training_features, test_features, training_target, test_target, = train_test_split(df.drop(['score_goal'], axis=1),
                                              df['score_goal'],
                                              test_size = .3,
                                              random_state=12)

clf_rf = RandomForestClassifier(n_estimators=25, random_state=12)
clf_rf.fit(training_features, training_target)
print("Accuracy using Random Forest Classifier is ", clf_rf.score(test_features, test_target)*100)

I'm confusing because I don't know how I can compare the predicted values to identify how many False Positives, etc. I have.

Anyone knows how can I build that function?

Thanks!

$\endgroup$
0
$\begingroup$

Looks like you're using scikit-learn. So why not explore a bit more? Scikit has a metrics module, that can be of use for your problem. Essentially, what you need is to have two separate arrays - one with real labels and another with predicted labels. And then you're good to go, you could call metrics.classification_report or metrics.confusion_matrix or metrics.accuracy_score, all of them use the real labels and the predicted labels.

There's nothing wrong with using clf_rf.score(test_features, test_target), but it will only give you a single value. If you look at the source code, what happens is that the score method calls a predict method with test_feature for prediction of labels, which occurs behind the scenes.

It's better to actually capture those predicted labels, so that you can reuse them.

clf_rf.fit(training_features, training_target)
predicted_target = clf_rf.predict(test_features)
accuracy = sklearn.metrics.accuracy_score(test_target, predicted_target)
cnf_matrix = sklearn.metrics.confusion_matrix(test_target, predicted_target)
class_report = sklearn.metrics.classification_report(test_target, predicted_target)

And then you could do whichever you like with the calculated metrics, print, plot, etc. Have a look at the examples that are included for each model/metrics you're using 1, 2, 3, 4, etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.