Does cross_val_score in scikit-learn split the data consistently or randomly? I noticed that cross_val_score lacks a random_state parameter, but the documentation mentions stratified k-fold cross-validation, which is implemented in the StratifiedKFold class that does have a random_state parameter for shuffling. So, how exactly does cross_val_score work? Does it split the data in a specific order or does it shuffle it? Furthermore, is the distribution of classes in each fold the same as the distribution in the original dataset? Lastly, which class or function in scikit-learn do you use for cross-validation?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
cross_val_score is a convenience function which relies on KFold or StratifiedKFold (see documentation):
When the cv argument is an integer, cross_val_score uses the KFold or StratifiedKFold strategies by default, the latter being used if the estimator derives from ClassifierMixin.
This implies that by default cross_val_score won't split the data randomly, since by default shuffle is false for both KFold and StratifiedKFold.
As per the documentation (see above), StratifiedKFold is used by cross_val_score if the classifier derives from ClassifierMixin. If so, the distribution of the classes will be strictly equivalent as the original distribution.
I tend to use KFold because it gives more control over what's being done.
