My organisation provides consultation to other firms, in part by making use of neural networks trained with extensive datasets that we have collected over the years.
Whenever available, we would inquire for any similar data our clients may have so that we can tailor a new model based on a checkpoint of our own. This has worked well as the data we used for training is relatively generalised and often collected by organisations in general.
However, recently one of our clients provided data that is highly predictive of the same response variable in our own models, but differs wildly in variable types and size.
An example in order to simplify:
I trained my own network to predict height based on weight using 100,000 records. Oftentimes my clients provide an additional 10,000 records of their employees height & weight which allows me to tailor the network to their needs. A new client, however, has provided me with 10,000 records of their employees' gender, country of origin, and daily dairy consumption. All of which, when trained in a separate network, prove highly predictive of height as well.
How do I effectively combine these separate models into a single classifier or regressor?