0
$\begingroup$

Recently I am working on the neural network deep learning algorithms, just curious to ask is it possible to merge two neural network models and to output one model that contains all the learned knowledge from the two models?

For example:

Model A can work on feature [A, B]

Model B can work on feature [C, D]

(Model type is the same for model A and model B if this makes things easier.)

After model merging, we get model C which is capable of working on feature [A, B, C, D].

I have looked into the transfer learning and Siamese network, but to my knowledge, I don't think these techniques can help me achieve my goal? (please correct me if I am wrong).

So any ideas will be much appreciated for merging the two models, or any possible technique, terminologies are welcomed.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The concept of Ensemble Learning is very applicable to your scenario: given two machine learning models that can predict well (or even just better than a random choice), combine the outputs of both models (with another ML model) to produce a unified model (let's call it Model C) that is more predictive than the individual models alone.

A straightforward application to your scenario could involve using the outputs of Models A and B along with some additional features from the original feature set as inputs to Model C. This larger set of features would allow for model C to learn for example when to the signal from Model A is more informative than B based on the current context provided by additional features from the original set.

It may also be possible to do this within a deep learning framework by adding both models into the same graph and adding a layer that ensembles the signals from the two models for an output prediction.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi grov, thanks for your answer, I am confused that if I use ensemble learning over different models that are trained on different dataset, for example now I want to process feature A, It will be enough to only use Model A for this case right? Model B here seems not helping as the result it generates is not accurate, right? I think a feature selector here will be much more suitable? For example: Feature A? -> Model A; Feature C? -> Model B. But this case doesn't help me to achieve the goal.. $\endgroup$
    – Elmer Wang
    Apr 4, 2021 at 0:48
  • $\begingroup$ When creating an ensemble model (C) to improve on two previously trained models (A & B), it is fine that models A and B were not trained on the same data. When training the ensemble model C, all that is required is the output predictions from models A & B on the dataset used to train C. $\endgroup$
    – grov
    Apr 5, 2021 at 2:06
  • $\begingroup$ Ensemble learning can improve performance in some cases (actually many common machine learning algorithms use ensemble learning to achieve good performance e.g. random forests). In some domains, feature selection may be important as well, especially if the dataset is small and there are many features. $\endgroup$
    – grov
    Apr 5, 2021 at 2:12
  • $\begingroup$ Thanks grov for the patient replies, appreciate it :) $\endgroup$
    – Elmer Wang
    Apr 6, 2021 at 7:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.