1
$\begingroup$

I am currently on a project that will build a model (train and test) on Client-side Web data, but evaluate this model on Sever-side Web data. Unfortunately building the model on Server-side data is not an option, nor is it an option to evaluate this model on Client-side data.

This model will be based on metrics collected on specific visitors. This is a real time system that will be calculating a likelihood based on metrics collected while visitors browse the website.

I am looking for approaches to ensure the highest possible accuracy on the model evaluation.

So far I have the following ideas,

  1. Clean the Server-side data by removing webpages that are never seen Client-side.
  2. Collect additional data Server-side data to make the Server-side data more closely resemble Client-side data.
  3. Collect data on the Client and send this data to the Server. This is possible and may be the best solution, but is currently undesirable.
  4. Build one or more models that estimate Client-side Visitor metrics from Server-side Visitor metrics and use these estimates in the Likelihood model.

Any other thoughts on evaluating over one Population while training (and testing) on another Population?

$\endgroup$
2
$\begingroup$

If the users who you are getting client-side data from are from the same population of users who you would get server-side data from. If that is true, then you aren't really training on one population and applying to another. The main difference is that the client side data happened in the past (by necessity unless you are constantly refitting your model) and the server side data will come in the future.

Let's reformulate the question in terms of models rather than web clients and servers.

You are fitting a model on one dataset and applying it to another. That is the classic use of predictive modeling/machine learning. Models use features from the data to make estimates of some parameter or parameters. Once you have a fitted (and tested) model, all that you need is the same set of features to feed into the model to get your estimates.

Just make sure to model on a set of features (aka variables) that are available on the client-side and server-side. If that isn't possible, ask that question separately.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I'm not an expert on this, so take my advice with a grain of salt. It's not clear for me what is the relationship between server-side and client-side data. Are they both representative of the same population? If Yes, I think it's OK to use different data sets for testing/training and evaluating your models. If No, I think it might be a good idea to use some resampling technique, such as bootstrapping, jackknifing or cross-validation.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.