Using scikit Learn - Neural network to produce ROC Curves

Question

I want to verify that the logic of the way I am producing ROC curves is correct. (irrelevant of the technical understanding of the actual code). I have a data set which I want to classify. I am using a neural network specifically MLPClassifier function form python's scikit Learn module.

I am passing a training data set to the fit function and then using the predict function with the testing data set. I am then outputting a confusion matrix with a false positive value and a true positive value.

what I would like to do is calculate a ROC curve where I need a set of true positive and false positive values. would it make sense to run the neural network (MLPClassifier) multiple times with different targets each time and record the different true positive and false positive values?

It seems simple enough for me. Is it the right way to produce a ROC curve with a neural network?

Answer 1

You could use sklearn.metrics.roc_curve.

Besides, Here is an example of what you want to do.

from sklearn.metrics import roc_curve, auc
fpr2, tpr2, threshold = roc_curve(y_test, clf.predict_proba(X_test)[:,1])