# How do we get the baseline of each fold in cross-validation?

Given that I am using Scikit learn and cross-validation and want to compare my accuracy result for each fold with my baseline

I am using 10-fold cross-validation and how for each fold I return the baseline for that fold. is it possible?

clf= RandomForestClassifier(n_estimators=100, random_state=20)
# 10-Fold Cross validation
scores = cross_val_score(clf, features, labels, cv=10)
scores


Result: array([0.45454545, 0.63636364, 0.8 , 0.8 , 0.6 , 0.6 , 0.5 , 0.9 , 0.66666667, 0.33333333])

I think you look for: sklearn.model_selection.cross_validate, see the docs here:

Returns:
scores : dict of float arrays of shape=(n_splits,) Array of scores of the estimator for each run of the cross validation. A dict of arrays containing the score/time arrays for each scorer is returned. The possible keys for this dict are:

test_score The score array for test scores on each cv split.

train_score The score array for train scores on each cv split. [...]

Minimal example:

from sklearn import datasets, linear_model
from sklearn.model_selection import cross_validate

# Data
X = diabetes.data[:150]
y = diabetes.target[:150]

# Model
lasso = linear_model.Lasso()
cv_results = cross_validate(lasso, X, y, cv=10)

# Print keys
print(sorted(cv_results.keys()))
# Print test_scores
print(cv_results['test_score'])


Returns:

['fit_time', 'score_time', 'test_score', 'train_score']
[ 0.34557351  0.34848715  0.26654262 -0.01126674  0.24875619  0.08731544
0.13386583  0.14000888  0.2873109   0.00960079]

• the test score is the accuracy result for each fold or the average of the label on each fold? – Guilherme Felipe Reis Jun 15 '19 at 10:14
• Read the docs: „The score array for test scores on each cv split“. Sounds to me like the score for each single step... – Peter Jun 16 '19 at 10:56

I found a solution, but I don't know if this is correct:

X = np.array(features)
y = np.array(labels)
kf = KFold(n_splits=10, random_state=20)
kf.get_n_splits(features)

print(kf)

for train_index, test_index in kf.split(X):
# print("TRAIN:", train_index, "TEST:", test_index)
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]
print(np.mean(y_test))


if a get the same randon_state, I expect to get the same folds.