I am working on a rare event (unbalanced target variable) classification problem using decision trees. My dataset comprises of 95% non-event and 5% minority (events) class.
I used decision tree over logistic regression because I had many categorical variables comparing to continuous variables. I get a good performance for training data with the decision tree C5.0. However I get poor results for the new data. I use the confusion matrix as a measure of performance. Training model is over-fitting.
I did pruning to reduce the over-fitting caused by the decision tree. I used the following code to build the model
Classifi_C5.0 <- C5.0(TARGET ~., , data = training_data_SMOTED, trails = 500, control = C5.0Control(minCases = mincases_count, noGlobalPruning = FALSE))
I balanced the minority and majority class using the following code:
training_data_SMOTED <- SMOTE(TARGET ~ ., training_data, perc.over = 100, k = 5, perc.under = 200)
Any sort of advice will be helpful.