1
$\begingroup$

I'm having a problem when using the k-fold cross-validation with the Random Forest method. One of the outputs is the error "Error in randomForest.default(x, y, mtry = param$mtry, ...) : Need at least two classes to do classification." However, I already have two classes to do the classification, which are "Normal" and "Failure". When posting this question at https://stackoverflow.com/questions/60643415/error-in-randomforest-defaultx-y-mtry-parammtry-need-at-least-two?noredirect=1#comment107290940_60643415 was recommended to me that I ask for "recommendations for statistical models given my data and your prediction/estimation/modeling needs".

Someone could help me?

The R script:

library(caret)    
library(randomForest)

data_failures <- read.csv('OUTPUT.csv', header = TRUE, sep = ",", stringsAsFactors = TRUE)

train.control <- trainControl(method = "cv", number = 10)
model <- train(Period_1 ~., data = data_failures, method = "rf",
                                   trControl = train.control)
print(model)

print(class(str(data_failures)))

The ouput:

Random Forest

112 samples
 11 predictor
  2 classes: 'Failure', 'Normal'

No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 101, 101, 101, 101, 101, 101, ...
Resampling results across tuning parameters:

  mtry  Accuracy  Kappa
   2    1         NaN
   6    1         NaN
  11    1         NaN

Accuracy was used to select the optimal model using the largest value.
The final value used for the model was mtry = 2.
'data.frame':   112 obs. of  12 variables:
 $ Period_1 : Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_2 : Factor w/ 2 levels "Failure","Normal": 2 2 2 1 2 2 2 2 2 1 ...
 $ Period_3 : Factor w/ 2 levels "Failure","Normal": 2 2 1 2 2 2 2 2 2 2 ...
 $ Period_4 : Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_5 : Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_6 : Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_7 : Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_8 : Factor w/ 2 levels "Failure","Normal": 2 2 2 1 2 2 2 2 2 2 ...
 $ Period_9 : Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_10: Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_11: Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
 $ Period_12: Factor w/ 2 levels "Failure","Normal": 2 2 2 2 2 2 2 2 2 2 ...
[1] "NULL"
Warning messages:
1: model fit failed for Fold08: mtry= 2 Error in randomForest.default(x, y, mtry = param$mtry, ...) :
  Need at least two classes to do classification.

2: model fit failed for Fold08: mtry= 6 Error in randomForest.default(x, y, mtry = param$mtry, ...) :
  Need at least two classes to do classification.

3: model fit failed for Fold08: mtry=11 Error in randomForest.default(x, y, mtry = param$mtry, ...) :
  Need at least two classes to do classification.

4: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.

A sample of the data:

    Period_1 Period_2 Period_3 Period_4 Period_5 Period_6 Period_7 Period_8
1     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
2     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
3     Normal   Normal  Failure   Normal   Normal   Normal   Normal   Normal
4     Normal  Failure   Normal   Normal   Normal   Normal   Normal  Failure
5     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
6     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
7     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
8     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
9     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
10    Normal  Failure   Normal   Normal   Normal   Normal   Normal   Normal
```
Improve this question
$\endgroup$
4
  • $\begingroup$ So what are you trying to predict exactly ? From your code you seems to try to predict what happen in period 1, from other periods. Is that your goal ? $\endgroup$
    – Lucas Morin
    Commented Mar 14, 2020 at 14:25
  • $\begingroup$ As Vincent said, not enough failure cases causes a random CV subset to contain 0 "failure" instances. You can also try to reduce the 'k' in k-fold btw. $\endgroup$
    – Erwan
    Commented Mar 15, 2020 at 1:00
  • $\begingroup$ @lcrmorin Exactly, I'm trying to predict what happens in the Period 1 from other periods. I want the accuracy of the model when predicting such results... $\endgroup$
    – Fernanda
    Commented Mar 15, 2020 at 18:01
  • $\begingroup$ @Erwan I've tried to reduce the "k" folds but it didn't work. The warning was: Warning message: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, : There were missing values in resampled performance measures. $\endgroup$
    – Fernanda
    Commented Mar 15, 2020 at 18:08

1 Answer 1

Reset to default
1
$\begingroup$

My guess is that either your Failure/Normal class is a lot lesser than the other. As such, for a certain (i.e. nth) fold, there only exists instances of one class. You can try doing oversampling the under-represented class to prevent this, or try doing a stratified K-Fold so that each fold will have occurrences of both classes.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I've tried to use the stratified k-fold, however, the output warning was: Warning message: In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo, : There were missing values in resampled performance measures. $\endgroup$
    – Fernanda
    Commented Mar 15, 2020 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.