I am using Random Forests, XGBoost and SVMs to classify whether the home team wins or the away team wins their bowl game (in college football). I trained the models on all the games during the season. I've come across something that is a bit weird and can't explain. I calculated a prediction confidence by subtracting the class probabilities. The XGBoost confidence values are consistency higher than both Random Forests and SVM's. I've attached the image below. I did some hyper-parameter tuning for all of my models and used the best parameters based on testing accuracy. - Random Forest: - 700 trees - 15 variables randomly sampled (mtries) - minimum split criteria of 5 rows. - XGBoost: - 0.5, Learn rate - gbtree as my booster - max depth of 6 - SVM: - RBF kernel - C (slack) of 1 - 0.01, Sigma [![First half][1]][1] [![Second half][2]][2] [1]: https://i.sstatic.net/WTCo5.png [2]: https://i.sstatic.net/aDyqz.png