https://www.cnblogs.com/guo-xiang/p/8044624.html explains with a nice example the mechanics of Nested Cross Validation.
In the picture, the example shows how to use Nested CV for hyperparameter tuning of the C parameter. Tee outer CV is run for K=3 folds and 2 folds CV in the inner CV.
This is my understanding (please feel free to correct me):
Step (1) In the first fold, we have P1, P2 as the outer training fold and P3 as the outer testing fold. Using P1 & P2, the inner CV kicks off for every hyperparameter. We get 2 average accuracies: 85% for C=1 and 80% for C=2. Since the Average accuracy for C=1 is the best, we use this hyperparameter setting and re-train on P1+P2. Evaluate on the P3 outer test fold to get 89% accuracy. We save the best score = 85% and best parameter C=1.
(2) Then, from the second fold, we have P1 & P3 as the outer training fold and P2. We get 2 average accuracies. It seems from the picture that the average accuracy for C=2 is the highest in the inner CV and so C=2 is selected. Using C=2, we re-train on P1+P3 and evaluate on P2 to get 84% accuracy. We save the best score from the inner CV and best parameter C=1.
(3) Repeat the same for the last outer third fold. We get C=1 from its inner fold and re-train on P2+P3; evaluate on P1 to get 76% accuracy.
Confusion: How to select the best hyper-parameter or which is the final model? After we come out of the outer CV loop, which one is the best hyperparameter? My understanding is that C=1 corresponding to 89% accuracy obtained from the first fold in Step 1 should be the best hyperparameter selected as that gives the highest accuracy amongst all the three outer folds. OR should the decision be made on the frequency?