0
$\begingroup$

I got the following code form a book on xgboost. I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes. mean accuracy is 81 which can be okay. but what if the training accuracy is 99% ? Shouldn't we also observe the training accuracy ? If yes, how can I do it since the model is fitted by the cross_val_score method with 5 difference cross validation-training sets ?

model = XGBClassifier(booster='gbtree', objective='binary:logistic', random_state=2) scores = cross_val_score(model, X, y, cv=5) print('Accuracy:', np.round(scores, 2)) print('Accuracy mean: %0.2f' % (scores.mean())) Accuracy: [0.85 0.85 0.77 0.78 0.77] Accuracy mean: 0.81

$\endgroup$

1 Answer 1

2
$\begingroup$

Try using cross_validate instead of cross_val_score, you need to set the parameter return_train_scorebool=True

You can refer to the documentation if you require further customization:

$\endgroup$
2
  • $\begingroup$ thank you for the response. I will try that! I can not upvote it as I am short of reputation points for doing so. $\endgroup$ Feb 19, 2023 at 17:43
  • $\begingroup$ thanks.. I hope this will help! $\endgroup$ Feb 20, 2023 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.