0
$\begingroup$

I've been running kfold cross validation with 10 folds and comparing it against a test set. Logging the score and the stdev along the way.

Once I wad happy with my model I then run the estimator through the cross validation with the full data (including the test set) but ... The standard deviation of the scores increases and the score decreased????

Is this normal? I assumed MORE data decreases variance ? Or is it just due to the randomness in the splits ?

Improve this question
$\endgroup$
1
$\begingroup$

The score decreasing and the standard deviation of the scores increases, as sample size increases, indicates overfitting to the smaller sample size.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I've tried everything to reduce overfitting... Can you suggest any good articles or have any suggestions for me? $\endgroup$ – Lewis Morris Feb 16 '20 at 15:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.