I am using Random Forests, XGBoost and SVMs to classify whether the home team wins or the away team wins their bowl game (in college football). I trained the models on all the games during the season.
I've come across something that is a bit weird and can't explain. I calculated a prediction confidence by subtracting the class probabilities. The XGBoost confidence values are consistency higher than both Random Forests and SVM's. I've attached the image below.
I did some hyper-parameter tuning for all of my models and used the best parameters based on testing accuracy.
- Random Forest:
- 700 trees
- 15 variables randomly sampled (mtries)
- minimum split criteria of 5 rows.
- 0.5, Learn rate
- gbtree as my booster
- max depth of 6
- RBF kernel
- C (slack) of 1
- 0.01, Sigma
I wasn't clear with my question: Why exactly does XGBoost prefer one class greatly to the other? In comparison to these other methods. I'm trying to figure out why my prediction confidences of a class are so high for XGboost.