Acceptable variation in accuracy of each k fold when using K-Fold Cross Validation?

I have a relatively small dataset consisting of 1432 samples.

I have trained a Random Forest Classifier and performed KFold CV. The results of running 10 Fold CV are as follows:

=== 10 Fold Cross Validation Scores ===

CVFold 1 = 90.2%
CVFold 2 = 87.6%
CVFold 3 = 86.7%
CVFold 4 = 86.7%
CVFold 5 = 83.9%
CVFold 6 = 75.8%
CVFold 7 = 87.2%
CVFold 8 = 82.8%
CVFold 9 = 86.1%
CVFold 10 = 89.3%

Mean Cross Validation Score:  85.6%


I am just not sure how to explain why there are such high variances between some of the folds, i.e from 75.8% in fold 6 to 90.2% in fold 1.

My understanding is that it is simply that the classifier found the samples in the 1st fold (90%) easier to classify than it did in fold 6 (75%) but I'm actually not entirely sure if that is the case.

I understand that each case is different but is it common to have such variances? And is it acceptable?

Edit: some details regarding my data

I have 5 classes, which are imbalanced:

class 1 - 5%

class 2 - 10%

class 3 - 15%

class 4 - 60%

class 5 - 10%

I am using SMOTE to balance the classes.

• The cross validation score variance can be quite high when the dataset has few observations. What's the size of yours (number of samples)? Dec 2 '19 at 21:12
• @RomainReboulleau I have a rather small dataset unfortunately with 1432 samples Dec 3 '19 at 9:59
• Please note, accuraccy is by far worst existing metric, you can base anything on. Please use RepeatedKFold with at least 50 cycles of training and testing. It's hard to get some insight into metrics distribution with just 10 values. Dec 3 '19 at 12:51
• What is your class balance? Dec 3 '19 at 12:51
• You apply resampling only to train split. I suggest you use stratified cv to ensure you maintain your balance when splitting. Dec 3 '19 at 13:27