How do we get the baseline of each fold in cross-validation?

Question

Given that I am using Scikit learn and cross-validation and want to compare my accuracy result for each fold with my baseline

I am using 10-fold cross-validation and how for each fold I return the baseline for that fold. is it possible?

clf= RandomForestClassifier(n_estimators=100, random_state=20)
# 10-Fold Cross validation
scores = cross_val_score(clf, features, labels, cv=10)
scores

Result: array([0.45454545, 0.63636364, 0.8 , 0.8 , 0.6 , 0.6 , 0.5 , 0.9 , 0.66666667, 0.33333333])

Answer 1

I think you look for: sklearn.model_selection.cross_validate, see the docs here:

Returns:
scores : dict of float arrays of shape=(n_splits,) Array of scores of the estimator for each run of the cross validation. A dict of arrays containing the score/time arrays for each scorer is returned. The possible keys for this dict are:

test_score The score array for test scores on each cv split.

train_score The score array for train scores on each cv split. [...]

---

Minimal example:

from sklearn import datasets, linear_model
from sklearn.model_selection import cross_validate

# Data
diabetes = datasets.load_diabetes()
X = diabetes.data[:150]
y = diabetes.target[:150]

# Model
lasso = linear_model.Lasso()
cv_results = cross_validate(lasso, X, y, cv=10)

# Print keys
print(sorted(cv_results.keys()))
# Print test_scores
print(cv_results['test_score'])

Returns:

['fit_time', 'score_time', 'test_score', 'train_score']
[ 0.34557351  0.34848715  0.26654262 -0.01126674  0.24875619  0.08731544
  0.13386583  0.14000888  0.2873109   0.00960079]

Answer 2

I found a solution, but I don't know if this is correct:

X = np.array(features)
y = np.array(labels)
kf = KFold(n_splits=10, random_state=20)
kf.get_n_splits(features)

print(kf)

for train_index, test_index in kf.split(X):
    # print("TRAIN:", train_index, "TEST:", test_index)
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    print(np.mean(y_test))

if a get the same randon_state, I expect to get the same folds.