How to select the learned model using $k$-fold cross validation?

Question

Let us consider a case where $1000$ data is given, i.e., the data set $U=\{x_1, \ldots, x_{1000}\}.$

When we want to use $k$-fold validation scheme, we first divide the data set into $k$ groups.

With out loss of generality, the parameter $k$ is assumed to be $10$.

Hence, we have $S_1=\{x_1, \ldots, x_{100}\}$, $S_2=\{x_{101}, \ldots, x_{200}\}$, $\ldots$, $S_{10}=\{x_{901}, \ldots, x_{1000}\}$.

I can obtain models, $f_k$, by learning a data set $U \setminus S_k$ for $k=1, 2, \ldots, 10$.

I can obtain error rates, $r_k$, by testing a data set $S_k$ with $f_k$.

Hence, I can obtain the error rates, $r$, by averaging $r_k$'s, i.e., $\sum_{k=1}^{10}r_k/10$.

I understand the $k$-fold cross validation so far.

---

Most materials I've seen just say the error rate averaged by $k$ scenarios in $k$-fold cross validation. However, they do no t say about $f_k$'s.

However, in this case, which model do I have to use among all $f_k$'s?

Answer 1

$k$-fold is just performed to obtain a measure of your accuracy, as using the training accuracy is generally a too optimistic measure of the accuracy. If you want to deploy a final model, what is recommended is to train a last model with all the data. In fact, when you compare two models, $f$ and $g$, what you do is obtain two error rates $r_f$ and $r_g$ by cross-validation, and you keep the model with the lowest error rate. After that, if the model with the lowest error rate is $f$, you retrain $f$ with all your data.

To sum up, $k$-fold cross-validation is a method to give measures of the performance of the model, if you want to obtain the best model just train it with all your data.