I am currently on a project that will build a model (train and test) on Client-side Web data, but evaluate this model on Sever-side Web data. Unfortunately building the model on Server-side data is not an option, nor is it an option to evaluate this model on Client-side data.
This model will be based on metrics collected on specific visitors. This is a real time system that will be calculating a likelihood based on metrics collected while visitors browse the website.
I am looking for approaches to ensure the highest possible accuracy on the model evaluation.
So far I have the following ideas,
- Clean the Server-side data by removing webpages that are never seen Client-side.
- Collect additional data Server-side data to make the Server-side data more closely resemble Client-side data.
- Collect data on the Client and send this data to the Server. This is possible and may be the best solution, but is currently undesirable.
- Build one or more models that estimate Client-side Visitor metrics from Server-side Visitor metrics and use these estimates in the Likelihood model.
Any other thoughts on evaluating over one Population while training (and testing) on another Population?