2
$\begingroup$

When a paper report the average and std of a model on a dataset, it means that they have changed the split of training and test sets and run the model multiple times or they just run the model on constant splitting multiple times to find the randomness of model? Which one is more common?

$\endgroup$
0
$\begingroup$

Usually, we report mean and variance for k-fold crossvalidation and similar techniques. We run the model multiple times in different data, but that can be applied to the second case too as, for example, a Neural Network can be initialized with random weights multiple times for test in the model.

If the paper does not explicitly say that they re-splited the dataset, then is more likely that this variance comes from random initialization. You should read the experimental procedures used and try to determine all sources of variance for papers results.

So it might be either or a combination of both dataset random splits and model random initialization

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.