Best method to deal with too many zeroes in regression problem?

Question

I'm trying to solve a simple regression problem using TensorFlow and Pandas to see what's the expected conversion rate for a given product given how much I'm willing to pay for each visit (cost per click or cpc).

Each data point represents a single day and its given acquisition cost and conversion rate. I expect conversion rate to decrease exponentially as visits increase because my audience gets wider and not as focused anymore.

The data seems to agree with this:

The problem is that for the majority of data points, the conversion rate is zero (i.e. no products were sold) so any model is likely to perform well by merely guessing zero sales. For this particular subset (one of the best selling products) the ratio of zero-to-non-zero is roughly 4:1 but in the general data it's about 300:1.

The two best ways that I can think of dealing with this are

a) summarizing data by time periods large enough to contain at least one conversion in them (problem with this is that I lose data by having to settle for the average cost instead of the more detailed daily data)

b) splitting the problem into a classification problem (will I sell at least one unit) and a regression problem (given that I'm going to sell at least one unit, how many units can I expect to sell)

Do either of those sound like a smart idea? Are there any tried-and-true methods to solve this kind of problem?

Answer 1

Option (b) is a good approach. If you use a classifier that outputs probabilities, you can even multiply the two model outputs together to calculate a risk expectation that's interpretable as the joint likelihood.

If cpc is discrete valued and non negative, you could interpret that as a count and use a zero-inflated Poisson model. If it's not, you could convert it by rounding to the nearest cent or something like that.

Answer 2

You asked about „tried-and-true“ method, thus I‘m suggesting the logistic regression.

I‘m not familiar with TF yet, so not sure whether this approach is easily doable there.

Below is the link to the answer explaining how to do it with R‘s glm: to put it shortly, besides specifying the response column (rates between 0 and 1) you have to provide the weight column representing the total number of trials.

how to do logistic regression for fractional outcome