1
$\begingroup$

How to print the accuracy in each fold of a Validation Dataset ? And how can I assign the fold number to each row in the dataframe?

classifier = RandomForestRegressor(n_jobs = -1, criterion='mse')
clf = GridSearchCV(classifier, param_grid = tunedParameters,cv=10)
all_accuracies = cross_val_score(classifier, X=X_train, y=y_train, cv=10)
print(all_accuracies) 
$\endgroup$
2
$\begingroup$

You're mixing up GridSearchCV and cross_val_score; you should only need to run one of them.

GridSearchCV will search through your hyperparameter space, for each combination using cross-validation and producing a score. You can access these scores through the attribute cv_results_.

cross_val_score has no hyperparameter search; it just scores using cross-validation. The output is a list of the individual fold scores.

If you've already used GridSearchCV, there's probably no reason to use cross_val_score. (After hyperparameter searching, you've seen and used all the data in that set, so the scores in cv_results_ are biased as would be the scores out of cross_val_score; if you need an unbiased estimate of performance, you'll need another test set (or nested cross-validation in the first place).)

If you want to keep track of which samples go in which fold, I think you need to use a cross-validation generator or iterable for the cv parameter instead of an integer. Then you can use that generator/iterable to also tell you which samples are in which fold.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.