Training And Testing Error Curves caret package in r

Question

I am Running the following models

1. Logistic regression
2. Decision Trees
3. SVM
4. Naive Bayes
5. Random Forest On the same data set.

I am using Caret package in r.

Its My dream to plot Training error and Testing error curves, I know those plots

are pretty beautiful as well as other performance measure curves using caret

package, any help would be really appreciated.

Answer 1

Here we are creating data to for a given model.

The main advantage with this code snippet is you are able to see how the model performers over various sizes of the training set. The learing_curve_data function can be found in the [6.0-7.1 version of caret][1]. Of course, learning curves are useful in machine learning for several reason which include comparison of algorithms, choosing model parameters, optimization, right data size to use for training.

set.seed(1412)
class_dat <- twoClassSim(1000)

set.seed(29510)
lda_data <- learing_curve_dat(dat = class_dat, 
                               outcome = "Class",
                               test_prop = 1/4, 
                               ## `train` arguments:
                               method = "lda", 
                               metric = "ROC",
                               trControl = trainControl(classProbs = TRUE, 
                                                        summaryFunction = twoClassSummary))


ggplot(lda_data, aes(x = Training_Size, y = ROC, color = Data)) + 
geom_smooth(method = loess, span = .8) + 
theme_bw()