Timeline for K-Fold cross validation-How to calculate regular parameters/hyper-parameters of the algorithms
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 30, 2021 at 2:16 | comment | added | Mohsen Sichani | I upvoted before as your answer and now accepted the answer. thanks once again. | |
| Jul 30, 2021 at 2:15 | vote | accept | Mohsen Sichani | ||
| Jul 30, 2021 at 2:15 | comment | added | Mohsen Sichani | Thanks a lot, @Erwan for spending time and explaining it throughly, really appreciate it. I have a better understanding of the CV Now. | |
| Jul 28, 2021 at 16:43 | comment | added | Erwan | Note that CV is not always needed, the standard way to train and test a model is to randomly split the data between a training set and a test set, train the model on the training set and evaluate on the test set. CV offers a slightly more reliable measure of performance but that's not always needed. And yes, it's used for tuning hyper-parameters for the same reason. | |
| Jul 28, 2021 at 16:40 | comment | added | Erwan | @MohsenSichani: CV is only meant to reliably evaluate a method, not to train the final model. It's a common confusion because it seems logical that since training happens during CV (k times), one could use one of the trained models as final model. But (1) this does not use the full data and (2) it doesn't make sense to pick the best model because this defeats the purpose of CV. the literature says that k-1 are used for training inside the CV process, it doesn't say that any of the CV models should be used as final model. | |
| Jul 27, 2021 at 15:48 | comment | added | Mohsen Sichani | thanks alot, Yes, I am talking about the regular parameter, not the hyperparams. So, those regular parameters (e.g. slope) are only obtained when the full dataset is used and then these obtained regular parameters are applied on the k-fold for finding the averaged error. Are you suggesting this? If this is the case then why in the literature it is suggested that k-1 fold in CV is used for Training and 1 fold for testing in each run? what is the purpose of this training? Based on your answer it seems k-fold CV is only used for identifying the performance of the hyper-parameter, right? | |
| Jul 27, 2021 at 10:20 | comment | added | Erwan | ... common method. Now to answer your question about the slope in a linear model: CV is used only for evaluation, not for actually producing the final model with its parameters. The final model (i.e. the values of the regular parameters) should simply be trained on the whole training set, it's not directly related to using CV. | |
| Jul 27, 2021 at 10:16 | comment | added | Erwan | @MohsenSichani no, the parameters themselves are not averaged. You might have a confusion between regular model parameters and hyper-parameters: the slope in a linear model is a regular parameter, as opposed to for instance the value of $k$ in $k$-nn. The difference is that the values of the hyper-parameters are not determined by the training (as opposed to regular model parameters), they must be decided another way and hyper-parameter tuning with CV is a ... | |
| Jul 27, 2021 at 3:21 | comment | added | Mohsen Sichani | Thanks @Erwan for the explanation, just a couple of questions, assume a 3-fold CV, in the first run (first fold) the slope is 2 (slope 2 gives the minimum of MSE), second run (second fold) slope is 3 and it gives the min MSE, and for the third run the slope is 4 (with min MSE), so the slope that appears as the result of this linear regression will be 3 ((2+3+4)/3) , right? does the algorithm will be run one more time on the whole dataset or 3 is the maximum number of runs? | |
| Jul 27, 2021 at 2:21 | history | answered | Erwan | CC BY-SA 4.0 |