I am always having cross validation score smaller then the training score and I am performing cross validation on just training data is that normal thing ? Kfold = 5 cross


1 Answer 1


Yes, it's called overfitting. Your model is beginning to memorize the training set, but not performing well on any validation or test set. If your question is why is this happening, I'd like to refer you to another answer I wrote explaining this phenomenon in more detail.

One interesting question that could be made is why is the performance on the cross-validation folds worse than on the test set?

This is a bit more tough to answer, because I don't have all the details. Some possible explanations could be that the since the training set is larger than each fold, the model was trained better, or that simply the test set examples happened to be easier.

  • $\begingroup$ cross validation is always performed on training data set right? $\endgroup$
    – zack
    Mar 24, 2020 at 0:08
  • $\begingroup$ Yes, that's right. $\endgroup$
    – Djib2011
    Mar 24, 2020 at 0:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.