I am working on developing a predictive model using Random Forest. There are a lot of users that log in to the site but only a fraction of them actually monetize on that day. I am trying to predict the probability of monetization of a user during a login. The predictor variables have been selected. Now the issue is that one user can log in on multiple days and have different values of predictor variables and the label (Monetized/ Not Monetized) for each log in.

If I try to train the data on the last 30 days' data, some users would occur multiple times and others would only occur one time. This may lead to a higher weight-age being given to the users who logged in on more days.

How can I ensure that equal weightage is assigned to all the users even though the users might have different numbers of logins and hence different counts of data points.

  • $\begingroup$ You could aggregate the training set by user over the training period. Then normalize your predictor variables, e.g. clicks/session as opposed to raw number of clicks. You could try this and just using your current data, which looks like it might be aggregated by session. Test both models with cross validation and confirm which aggregation is better. $\endgroup$ – user13684 Nov 24 '15 at 13:36
  • $\begingroup$ If the dataset reflects the true distribution of logins, wouldn't you be more concerned with balancing between positive/negative examples rather than individual users? $\endgroup$ – jamesmf Nov 24 '15 at 13:52
  • $\begingroup$ @jamesmf , Well the Login to monetization ratio is already at around 70% so the number of positives are already pretty high enough I think. Its not really like a case of say something like a credit card fraud with positives in the range of 2%-3% only $\endgroup$ – Vaibhav Srivastava Nov 24 '15 at 16:26
  • $\begingroup$ @init-random Yes, the current data is aggregated on a daily basis (Whether the user monetized on the same day that he logged in). The predictor variables that are strong learners (based on the success some of the other similar models) are derived out of the user activity of the previous 7 days. Since the day of the week of the visit is also a very important predictor, I am out of ideas on how to aggregate this by user. Following your cue, I'll try to make 7 models, one for each weekday. Then this could be aggregated. $\endgroup$ – Vaibhav Srivastava Nov 24 '15 at 16:39
  • $\begingroup$ @Vaibhav Srivastava Balancing your data doesn't just mean 'having enough positives.' But my question is why are you aggregating? If your goal is to predict a the probability of monetizing per login, you should train on per-login information. You can easily include historical data (7 day activity, for example) for each login. But if you aggregate during training, you lose the granularity of per-login. $\endgroup$ – jamesmf Nov 24 '15 at 16:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.