0
$\begingroup$

My organisation provides consultation to other firms, in part by making use of neural networks trained with extensive datasets that we have collected over the years.

Whenever available, we would inquire for any similar data our clients may have so that we can tailor a new model based on a checkpoint of our own. This has worked well as the data we used for training is relatively generalised and often collected by organisations in general.

However, recently one of our clients provided data that is highly predictive of the same response variable in our own models, but differs wildly in variable types and size.

An example in order to simplify:

I trained my own network to predict height based on weight using 100,000 records. Oftentimes my clients provide an additional 10,000 records of their employees height & weight which allows me to tailor the network to their needs. A new client, however, has provided me with 10,000 records of their employees' gender, country of origin, and daily dairy consumption. All of which, when trained in a separate network, prove highly predictive of height as well.

How do I effectively combine these separate models into a single classifier or regressor?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

This process is called fine tuning frequently in the neural network literature. If you already have the trained network, you simply run additional training steps on the new data.

You'll need to balance the weight of the prior training with that of the new observations to some degree, and that decision is parameterized by the number of free variables in the network, the learning rate of the new training steps, and the number of new training steps.

However, your particular case is very complicated by the presence of new variables. Maintaining the form of the prior network is difficult in this case. You may want to simply ensemble a newly trained network on the new data and variables with the old network to combine the predictions.The simplest ensemble (but maybe not the best) would be averaging the results of the two models.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the tip. I'm currently experimenting with ensembles but the general theme behind it seems to be that input variables have to be the same or that they take missing values into account accordingly (and labeling all off the mismatched records as missing won't do the model any good). $\endgroup$
    – Rodrigo Perotto
    Commented Nov 1, 2018 at 8:32
  • $\begingroup$ You could definitely take the average prediction of two models that use different variables to make those predictions. That would be the simplest working ensemble for this problem. $\endgroup$
    – Thomas Cleberg
    Commented Nov 1, 2018 at 11:35
  • $\begingroup$ Awesome. I'm starting with that right away. Thanks for your help! $\endgroup$
    – Rodrigo Perotto
    Commented Nov 1, 2018 at 12:16
  • $\begingroup$ If you found the answer useful, it helps others who have a similar question if you upvote it. If it completely answers your question, accepting the answer is how you show that. Welcome to Stack Overflow! $\endgroup$
    – Thomas Cleberg
    Commented Nov 1, 2018 at 13:06
0
$\begingroup$

You could use an ensemble technique to combine the results of the two trained neural networks based on an optimized weighting. This could be as simple as passing the results into a new neural net or by using sklearn.ensemble.VotingClassifier

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the reply. I will experiment with this idea of passing the predictions of both models onto a third one as inputs. This however does raise some other questions: Do I train this third model as well, or tweak weights manually (for instance, based on MSE). If I do train it, does it imply that I need to collect new data involving rows that have all of the data, or feed my general bulk of data in with the new variables labeled as missing, and vice versa? $\endgroup$
    – Rodrigo Perotto
    Commented Nov 1, 2018 at 8:34
  • $\begingroup$ The sklearn.ensemble.VotingClassifier can perform equal weight voting. I assume the weights could be supplied. Then you just need to determine the weights. Since you are already using NN, I would probably just join the two result classes into a new record with the actual class and send it into a third NN. $\endgroup$
    – Skiddles
    Commented Nov 1, 2018 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.