0
$\begingroup$

I have an imbalanced dataset. Does it make sense to compute the roc-auc for the classifier I created in a holdout set?

Here's very artificial MWE:

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
clf = LogisticRegression(solver="liblinear").fit(X, y)

# Let's assume that X_test = X, y_test is just a vector of 1s. 
roc_auc_score([1]*150, clf.predict_proba(X_test), multi_class='ovr')

ValueError: Number of classes in y_true not equal to the number of columns in 'y_score'
$\endgroup$
1
$\begingroup$

predict_proba method will return a numpy array of shape (n_samples,2) with the probability of Y == 1 and Y == 0 but you need to pass only the probability of Y == 1 for roc calculation so:

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
clf = LogisticRegression(solver="liblinear").fit(X, y)

# Let's assume that X_test = X, y_test is just a vector of 1s. 
roc_auc_score([1]*150, clf.predict_proba(X_test)[:,1], multi_class='ovr')
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.