I'm using Repeated Stratified k-fold cross-validation and Oversampling to deal with imbalanced classes in a binary classification problem, using the ROC AUC metric. My question is: Is it possible to estimate and save specificity and sensitivity during training?

# define pipeline
steps = [('over', RandomOverSampler()), ('model', LogisticRegression())]
pipeline = Pipeline(steps=steps)
# evaluate pipeline
cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=1)
scores = cross_val_score(pipeline, X, Y, scoring='roc_auc', cv=cv)
score = mean(scores)
print('ROC Oversampling: %.4f' % score)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.