How to combine two different machine learning models, to get the combined result?

Question

To further explain my question, I will explain my use-case. Say I have a model which is trained for how good/bad a food is for obesity based on its nutrition facts. And another model for, say hypertension. I wish to combine these models to be able to predict food which are good for a person suffering from both obesity and hypertension.

I do not wish to retrain a new model for both cases as eventually I will add more diseases and do not wish to train for each combination of diseases.

Ideally I would like to be able to combine different models (eg Logistic Regression, SVM) but as a stepping stone, I was looking to combine two models of the same type.

I am not sure if Stacking would be appropriate in this case.

Answer 1

Train another XGB model on second layer. Your first layer contains all the individual model and in the second layer you will train another model on top of all of them. XGB is good for this task.

Answer 2

Ensembling and stacking are made to improve the model's predictive capacity. But your need here is to enable the model to predict a new Class.

This doesn't happen in machine Learning unless you use your domain knowledge and infer that from the individual prediction.
e.g. If I train a model to predict white color and another to train Black color, it doesn't mean it can predict grey color. For that -
I need to train the model on grey color
or deduce from the two predictions e.g. If probability ~0.6 for both then it might be grey etc.

In your case -
- Train on a dataset which have data for "Diabetes", "Hypertension", "Both", "None" classes
- Or put some logic on the probability of the 2 models e.g. Individually good for both disease means good for a patient which is suffering from both. But I think, this will be a very big assumption.