3
$\begingroup$

I am new to machine learning. I have a task at hand of predicting click probability given user information like city, state, OS version, OS family, device, browser family, browser version, etc. I have been advised to try logit since logit seems to be what MS and Google are using. I have some questions regarding logistic regression:

Click and non click is a very very unbalanced class and the simple GLM predictions do not look good. How can I make the data work better with the GLM?

All the variables I have are categorical and things like device and city can be numerous. Also the frequency of occurrence of some devices or some cities can be very very low. How can I deal with this distribution of categorical variables?

One of the variables that we get is device ID. This is a very unique feature that can be translated to a user's identity. How can I make use of it in logit, or should it be used in a completely different model based on user identity?

Improve this question
$\endgroup$
6
  • $\begingroup$ That is a lot of open ended questions to ask all at once. One of the goals of this site is that questions and their answers should be useful to future visitors. Can you be more specific? What looks bad in your linear model? When you tried the logistic model, how did it come out? Were there specific problems? $\endgroup$
    – Air
    Commented Jun 30, 2014 at 15:52
  • $\begingroup$ Sorry,an eg: On trying logit in R with these values: day-28, hour-11,day of the week-7, state-New Mexico, OS-iOS, OS Version-7.0, browser family-Mobile Safari, browser version-7.0, device manufacturer-apple, IP carrier- Comcast, user age-20, gender-male, click happened-yes, predicted probability from GLM in R-0.000000001. In another instance,day-28, hour-12th, day of the week-7, state-Connecticut, OS-iOS, OS version-7.0, browser-mobile safari, browser version-7.0, device manufacturer-apple, IP carrier-Comcast, user age-22, user gender-female,click happened-no, predicted probability-.046 $\endgroup$
    – aasthetic
    Commented Jun 30, 2014 at 17:02
  • $\begingroup$ Thanks. There's an "edit" link at the bottom of your question so that you can update it with this additional information. $\endgroup$
    – Air
    Commented Jun 30, 2014 at 17:13
  • $\begingroup$ That is what I meant by how the model does not look good. I think this is due to the sparsity of instances of clicks happening and random nature of event of clicks happening. So how can I go about sampling the data? Also how should I add weights to the parameters for improving the prediction? Sorry for adding more open-ended questions but encountering issues as they come, I also want to ask how to work with large datasets in R? $\endgroup$
    – aasthetic
    Commented Jun 30, 2014 at 17:44
  • $\begingroup$ The class imbalance is such that you don't really want to predict click vs no click. Since almost no one clicks your best result is to simply predict they don't click. Instead, think about predicting for all the users that click...which ad will they click on. This is a much more useful problem. $\endgroup$
    – cwharland
    Commented Jul 3, 2014 at 3:49

1 Answer 1

Reset to default
2
$\begingroup$

Some suggestions

  1. Remove items appearing too infrequently in the data. That will reduce the dimensionality by several orders of magnitude. If a feature occurs less than say 10 times, it's likely that it's not adding any predictive value, and it may lead to overfitting due to low frequency
  2. Try a Linear SVM instead. They handle large dimensional data very well in terms of not overfitting. They also often have the option to assign relative weights to different classes, which may help address your unbalanced problem above. The sklearn svm (which simply wraps some other packages such as libsvm) has this option.
  3. Don't use the ID column. Producing a model per user will most probably lead to overfitting. Instead, feed in attributes that describe the user that allows the model to generalize over similar users. You could try fitting a separate model per user, but you need a lot of data per user to do this well.
  4. It sounds like you really need to try some feature selection here, to reduce the dimensionality of the problem. But try 1 and 2 first, as they may give you good results sooner (although the end solution may still work better with some good feature selection). Sklearn again has a number of options for feature selection.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.