Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
I have to train a classification model to predict if a customer will buy a product or not. I have multiple (eg. 3 or 4) data sources. The variable distributions among the different data sources is quite different (eg. in the first one I have a vast majority of young people, while in the second one there are many adults). When the model will be used in production, the test dataset will be made of records from only one data sources (I don't know in advance which one).
My question is: which is the best way to combine these different sources? The data are not so much, so I cannot train an independent model for each data source. Can I just concatenate the data frame or do I have to perform some other kind of step?
Add all data together, and make sure you have features representing all possible insights. In your case one feature with age/maturity (young, adults...). Lets say you fit a decision tree (or Random Forest, gradient boosting...) the model will decide whether to do a split or not on this feature if it contains meaningful information.
If you combine you should be able to have more data and the models will work better.
In case you have different columns in data1 than in data2, consider leaving them as NaN, and then treating them properly.
The basic one, club the whole data and try different algorithms and evaluate the results.
If Age distribution among the different samples(data set) are not proportionally distributed, i.e. if your dataset have huge samples of 'Young' in comparison to 'Adult' one or vice-versa, then I definitely try to tune my model without considering 'Age' as a feature. Trained model can be biased towards majority class, that's why good to drop such features.
Ensemble based approach can also be explore. Train model for 'young' and 'adult' group separately. During testing, instead of giving equal score's to each model in ensemble, I will decide the contribution factor based upon the age values.