The Goal Compare several preprocessing methods and models - while tuning hyperparameters for each model - without leaking information into the final generalization estimate, applying cross-validation (cv), not fixed train/test splits.

Not the problem: The principles of cross-validation, nested cv, fixed train/val/test splits. With fixed train/val/test splits it is easy but I thought fixed splits are too risky, or are they not (hyperparameter tuning with validation set)?

Previous attempts, proposed solutions: So far I was not able to get the precise order of procedures from previous solutions. Although 1-liner packackage solutions may be the final best practice, they are not helpful to explain what happens. Better would be pseudo-code schemes as below

I also could not extract the precise order from these threads:



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy