2
$\begingroup$

I need to try out few different machine learning methods (SVM, Logistic regression etc.), predict a value either true or false, and write down their AUC and Accuracy of these predictions. I have allready successfully done that, now i have a two matrixes one for AUC and one for Accuracy, and they are filled with data from SVM and logistic regression (one row).

Now i need to create models for SVM and Logistic regression 10 more times (i should use bootstrapping sampling) and with that i should have 10 rows of my AUC and accuracy data. I have read multiple articles and guides/tutorials, however i can't figure out how to achieve this. I also found and tried few libraries ( one is ROSE and the other one is boot) and none worked for me. Because if i understand the assigment correctly i need to get 10 different samples from my dataset, and then seperate the data in train and test sets so i can compare the models AUC and accuracy and see how good those models actually are.

Like i said i found multiple sources and the best thing i came up with is this:

 for (i in 1:10){
      set.seed(123)
      ##########################
      ##########################
      boot.sample = sample(n, 1000, replace = TRUE)
      bootSample = dataset[boot.sample, ]
      bootSample
    
      split = sample.split(bootSample$blueWins, SplitRatio= 0.80)
      training = subset(bootSample, split == TRUE,  replace=TRUE)
      test = subset(bootSample, split == FALSE,  replace=TRUE)
      print(training)
}

But with this approach i think set.seed messes up everything, because it works with the same data every time. However i think the assingment wants me to use the same seed for every machine learning model.

I maybe overcomplicated the whole thing, i am new to R.

Hope someone can clear these things up. Thanks

$\endgroup$

2 Answers 2

1
$\begingroup$

Try using a different seed for each loop. You can do it like this.

my_seeds <- c(1:10) # These are 10 seeds, 1, 2, 3...10. Change to whatever.     
for (i in 1:10){
          set.seed(my_seeds[i])
          ##########################
          ##########################
          boot.sample = sample(n, 1000, replace = TRUE)
          bootSample = dataset[boot.sample, ]
          bootSample
        
          split = sample.split(bootSample$blueWins, SplitRatio= 0.80)
          training = subset(bootSample, split == TRUE,  replace=TRUE)
          test = subset(bootSample, split == FALSE,  replace=TRUE)
          print(training)
    }
$\endgroup$
0
$\begingroup$

You can set seed once outside the loop:

set.seed(123)
 for (i in 1:10){
      
      ##########################
      ##########################
      boot.sample = sample(n, 1000, replace = TRUE)
      bootSample = dataset[boot.sample, ]
      bootSample
    
      split = sample.split(bootSample$blueWins, SplitRatio= 0.80)
      training = subset(bootSample, split == TRUE,  replace=TRUE)
      test = subset(bootSample, split == FALSE,  replace=TRUE)
      print(training)
}
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.