# How to calculate y_score for ROC AUC?

how can I calculate the y_score for a roc_auc_score?

I have a classifier, for classes {0,1}, say RandomForestClassifier.

Then, when I apply it to my test data, I will get a list of {0,1}

But roc_auc_score expects y_true and y_score.

As dummy as it might look, after fitting the model, I was making the following:

def find_prob(df):
def prob(token):
# the mean is the probability of label {1}
return df[df.token == token].predicted.mean()
df['predicted_score'] = df.token.map(prob)
return df

X_val['predicted'] = model.predict(X_val).astype(int)
X_val = find_prob(tokenize(X_val))

roc_auc_score(y_val, X_val.predicted_score)


where tokenize is only a function which concatenate all the values in a row to a "unique" token string, so that I would be able of finding all instances of a token and check how many were labeled as {0} and how many were labeled as {1}.

Is this the correct approach? Thank you very much.

y_score can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions

y_score = model.predict_proba(x)[:,1]

AUC = roc_auc_score(y, y_score) # Above 0.5 is good


So, I have found my answer, Now I am going to share it. y_score is the prediction of the x_test, where you can say in my dataset, it is images, in dataset_train. You can get the prediction of the dataset using this -

p = torch.nn.functional.softmax(output, dim=1)
prediction = torch.argmax(p, dim=1)


Where p is the probability and then using argmax, you can get what you want.