1
$\begingroup$

Given that I am using Scikit learn and cross-validation and want to compare my accuracy result for each fold with my baseline

I am using 10-fold cross-validation and how for each fold I return the baseline for that fold. is it possible?

clf= RandomForestClassifier(n_estimators=100, random_state=20)
# 10-Fold Cross validation
scores = cross_val_score(clf, features, labels, cv=10)
scores

Result: array([0.45454545, 0.63636364, 0.8 , 0.8 , 0.6 , 0.6 , 0.5 , 0.9 , 0.66666667, 0.33333333])

$\endgroup$
1
$\begingroup$

I think you look for: sklearn.model_selection.cross_validate, see the docs here:

Returns:
scores : dict of float arrays of shape=(n_splits,) Array of scores of the estimator for each run of the cross validation. A dict of arrays containing the score/time arrays for each scorer is returned. The possible keys for this dict are:

test_score The score array for test scores on each cv split.

train_score The score array for train scores on each cv split. [...]


Minimal example:

from sklearn import datasets, linear_model
from sklearn.model_selection import cross_validate

# Data
diabetes = datasets.load_diabetes()
X = diabetes.data[:150]
y = diabetes.target[:150]

# Model
lasso = linear_model.Lasso()
cv_results = cross_validate(lasso, X, y, cv=10)

# Print keys
print(sorted(cv_results.keys()))
# Print test_scores
print(cv_results['test_score'])

Returns:

['fit_time', 'score_time', 'test_score', 'train_score']
[ 0.34557351  0.34848715  0.26654262 -0.01126674  0.24875619  0.08731544
  0.13386583  0.14000888  0.2873109   0.00960079]
| improve this answer | |
$\endgroup$
  • $\begingroup$ the test score is the accuracy result for each fold or the average of the label on each fold? $\endgroup$ – Guilherme Felipe Reis Jun 15 '19 at 10:14
  • $\begingroup$ Read the docs: „The score array for test scores on each cv split“. Sounds to me like the score for each single step... $\endgroup$ – Peter Jun 16 '19 at 10:56
0
$\begingroup$

I found a solution, but I don't know if this is correct:

X = np.array(features)
y = np.array(labels)
kf = KFold(n_splits=10, random_state=20)
kf.get_n_splits(features)

print(kf)

for train_index, test_index in kf.split(X):
    # print("TRAIN:", train_index, "TEST:", test_index)
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    print(np.mean(y_test))

if a get the same randon_state, I expect to get the same folds.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.