Estimating expected revenue generation

Question

I have a number of features X and a target which is the revenue generated by this interaction. More often than not no revenue is generated, but if there is it can differ considerably, it has a big right tail. I've made a model to estimate the expected revenue generated by a new x. Due to the weird distribution of the revenues I have split the model up in two parts, first estimating the probability of revenue generation and then conditionally on the fact that somebody will book, predict the amount of $log($revenue$)$. The first model is trained on a dummy variable which indicates revenue and the second one is trained on a subset of the data which contains all interactions that generated revenue.

After I have these two models P (probability of generation) and R (expected revenue generated conditioned on > 0) we can calculate the expected revenue for a new x using:

$$f(x) = P(x)e^{R(x)}$$

I've done a reasonable amount of cross validation on both the models individually and they work decent, however when they are combined they severly underestimate the amount of revenue generated. I have a good enough number of rows for the dimensionality of my data and I'm using Logistic Regression for the probability (linearity assumption is somewhat reasonable, however unbiased probabilities are very important) and Gradient Boosting for revenue estimation (got the best results for the second portion).

What should I do about the underestimation of the revenue? What are alternative unbiased probablity estimators, should I train a neural network with a hidden layer for the log-loss function? Why does the combined model underestimate? Is there a good alternative to do it in one model? Which would not increase small errors into huge errors due to the multiplication and exponent?

Answer 1

Why not just use quantile regressions?

Usually, in regressions, the coefficients are estimated for the average case: $\min_\beta \sum_i||X_i\cdot\beta -y_i||^2$.

If you want, you can estimate your coefficients so that they tell you things like $P(R=y)>p$. You give a probability $p$, and the model gives you $y$. You can even build probability density function out of these estimation techniques.

In quantile regressions, you solve this optimization problem for every quantile $\tau$ that you want, $\min_\beta \sum_{i|y_i\geq X_i\cdot\beta}\tau|X_i\cdot\beta-y_i| + \sum_{i|y_i< X_i\cdot\beta}(1-\tau)|X_i\cdot\beta-y_i|$.

This is a more robust approach to modeling the revenue distribution. Usually people use quantile regressions because software is readily available, but they can be implemented for other models as well. I work in an institute that does exactly that.