Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
What are some possible techniques for smoothing proportions across very large categories, in order to take into account the sample size? The application of interest here is to use the proportions as input into a predictive model, but I am wary of using the raw proportions in cases where there is little evidence and I don't want to overfit.
Here is an example, where the ID denotes a customer and impressions and clicks are the number of ads shown and clicks the customer has made, respectively.
A simple way would be to consider Laplace Smoothing (http://en.wikipedia.org/wiki/Additive_smoothing ) or something like it.
Basically, instead of calculating your response rate as (Clicks)/(Impressions) you calculate (Clicks + X)/(Impressions + Y), with X and Y chosen, for example, so that X/Y is the global average of clicks/impressions.
When Clicks and Impressions are both high, this smoothed response rate is basically equal to the true response rate (signal dominates the prior). When Clicks and Impressions are both low, the this smoothed response rate will be close to the global average response rate - a good guess when you have little data and don't want to put much weight on it!
The absolute scale of X and Y will determine how many data points you consider "enough data". It's been argued that the right thing to do is set X to 1, and Y appropriately given that.
