I am very new to machine learning and in my first project have stumbled across a lot of issues which I really want to get through.
I'm using logistic regression with R's
glmnet package and alpha = 0 for ridge regression.
I'm using ridge regression actually since lasso deleted all my variables and gave very low area under curve (0.52) but with ridge there isn't much of a difference (0.61).
My dependent variable/output is probability of click, based on if there is a click or not in historical data.
The independent variables are state, city, device, user age, user gender, IP carrier, keyword, mobile manufacturer, ad template, browser version, browser family, OS version and OS family.
Of these, for prediction I'm using state, device, user age, user gender, IP carrier, browser version, browser family, OS version and OS family; I am not using keyword or template since we want to reject a user request before deep diving in our system and selecting a keyword or template. I am not using city because they are too many or mobile manufacturer because they are too few.
Is that okay or should I be using the rejected variables?
To start, I create a sparse matrix from my variables which are mapped against the column of clicks that have yes or no values.
After training the model, I save the coefficients and intercept. These are used for new incoming requests using the formula for logistic regression:
a is intercept,
k is the
ith coefficient and
x is the
ith variable value.
Is my approach correct so far?
Simple GLM in R (that is where there is no regularized regression, right?) gave me 0.56 AUC. With regularization I get 0.61 but there is no distinct threshold where we could say that above 0.xx its mostly ones and below it most zeros are covered; actually, the max probability that a click didn't happen is almost always greater than the max probability that a click happened.
So basically what should I do?
I have read how stochastic gradient descent is an effective technique in logit so how do I implement stochastic gradient descent in R? If it's not straightforward, is there a way to implement this system in Python? Is SGD implemented after generating a regularized logistic regression model or is it a different process altogether?
Also there is an algorithm called follow the regularized leader (FTRL) that is used in click-through rate prediction. Is there a sample code and use of FTRL that I could go through?