Suppose you have a classification task y~X with (n_samples,m_features). A colleague told me that it is correct to run r different classifiers to predict y based on X and then use the probabilities given for each classifier as a new matrix Xnew (n_samples_probabilities,r_columns) to train a new classifier y~Xnew

My questions are:

1) Is this something reasonable to apply?

2) If so, is there any mathematical support on this method?


This is called "stacked ensembling," or just "stacking."


I'm not aware of any theoretical mathematical support, but intuitively it may smooth out the results of the $r$ initial models, possibly (depending on the final model) promoting models on data segments where they perform best. Stacking consistently wins Kaggle competitions, though the added complexity might make it less attractive for certain applications.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.