cv-mse plot

Be gentle, I'm learning here. I have a fairly simple adaptive lasso regression that I'm trying to test for a minimum sample size. I used cross-validated mean squared error as the "score" of model accuracy. Where I am stuck is how to analyze each group of samples to determine at what point the CV-MSE stops being significantly different from the last smaller size. I believe the tactic is good, or maybe not; please tell me. But I am just stuck on how to decide which sample size to select.

The figure is a box plot with jitter, each dot representing one test. Cross-validated MSE is on the Y-axis, and the simulation's sample size is on the X-axis.

Just for clarity, this is just a one-time regression model using alasso with predictor and response variables. I'm not training a model for testing. It "looks" like the CV-MSE levels out around 375-400 cases. What I'm asking here is, is there a better way than eye-balling the plot to determine the optimal sample size I need to collect to run the test?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.