2
$\begingroup$

When I am doing cross validation using Python's Sklearn and take the score of different metrics (accuracy, precision, etc.) like this:

result_accuracy = cross_val_score(classifier, X_train, y_train, scoring='accuracy', cv=10)
result_precision = cross_val_score(classifier, X_train, y_train, scoring='precision', cv=10)
result_recall = cross_val_score(classifier, X_train, y_train, scoring='recall', cv=10)
result_f1 = cross_val_score(classifier, X_train, y_train, scoring='f1', cv=10)

Did every execution of cross_val_score() function for different metrics made the same 10 folds of the training data or not? If not, do I need to make the general 10-folds first using KFold, like this:

seed = 7
kf = KFold(n_splits=10, random_state=seed)

result_accuracy = cross_val_score(classifier, X_train, y_train, scoring='accuracy', cv=kf)
result_precision = cross_val_score(classifier, X_train, y_train, scoring='precision', cv=kf)
result_recall = cross_val_score(classifier, X_train, y_train, scoring='recall', cv=kf)
result_f1 = cross_val_score(classifier, X_train, y_train, scoring='f1', cv=kf)
$\endgroup$
2
$\begingroup$

It only really matters if you want to shuffle your data in the cross-validation. The default for both cross_val_score and KFold is to NOT shuffle.

If you do want to shuffle your second option is best if you want to make sure that you are comparing scores across the same splits on the data.

Keep in mind that in the default KFold(shuffle=False), so if you want to shuffle your data make sure that you set Kfold to:

kf = KFold(n_splits=10, shuffle=True, random_state=seed)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.