1
$\begingroup$

I saw on a CatBoost site that it supposed to outperform any other boosted training model and decided to try it myself on a Kaggle's https://www.kaggle.com/c/house-prices-advanced-regression-techniques.

I created some basic kernel without any complex preprocessing, feature selection, GridSearch, stacking, etc... just to compare the XGBR and CatBoost performance. But as far as I see the XGBR always outperforms the CatBoost.
https://www.kaggle.com/markbquant/compare-catboostregressor-vs-xgbregressor

The CatBoost parameters: iterations=100, depth=3, learning_rate=0.1
The XGBR parameters: subsample=0.7, colsample_bytree=0.7, n_estimators=500, learning_rate=0.03, max_depth=5, min_child_weight=3

For example, inside the Kaggle, the XGBR received score 0.134 and the CatBoot only reached 0.197 (I tried both one_hot_max_size and cat_features). I'll be thankful if someone could point me what is wrong with the CatBoost model, may be some optimization is missing.

$\endgroup$
1
  • $\begingroup$ So on how many different datasets / tasks did you do your comparison experiment? 1? $\endgroup$
    – Isbister
    Nov 29 '18 at 18:59
2
$\begingroup$

Performing such benchmark is not that easy. Meaning one can not just pick a few data set and run these models as there is a data dependency. In such cases, one need to simulate data through various process - the simulation helps to design various data in various condition. for example perhaps model one is doing a better job at binning so, the data with various binning condition must be in place before hand, or the depth of the tree. So just picking house-prices data is not enough.

Bare in mind such out performance can be really really small. Don't expect 10% difference ! they lay down within 1% often.

What distinguish Catboost from Xgboost is the thread safing in the production environment. Xgboost is not a thread safe - therefore, can not be used in any serious deployment environment.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.