# Proper way of fighting negative outputs of a regression algorithms where output must be positive all the way

Maybe it is a bit general question. I am trying to solve various regression tasks and I try various algorithms for them. For example, multivariate linear regression or an SVR. I know that the output can't be negative and I never have negative output values in my training set, though I could have 0's in it (for example, I predict 'amount of cars on the road' - it can't be negative but can be 0). Rather often I face a problem that I am able to train relatively good algorithm (maybe fit a good regression line to my data) and I have relatively small average squared error on training set. But when I try to run my regression algorithm against new data I sometimes get a negative output. Obviously, I can't accept negative output since it is not a valid value. The question is - what is the proper way of working with such output? Should I think of negative output as a 0 output? Is there any general advice for such cases?

• would you consider a function which more accurately reflects the data you have rather than a linear regression model ? Feb 1, 2015 at 13:26
• I tried different models, but so far all of them sometimes output negative values. This is not really wrong - if you think of linear regression as a line it is possible that it is not able to build a function which will be ALWAYS positive, or even carefully 'touch' the axis (to output 0s). But once again, whatever I tried, for new data I sometimes keep getting negatives. Feb 1, 2015 at 16:39
• Hard to know for definite without sight of the data, but one could imagine a piecewise linear approximation that could be shaped not to go negative, or a high order polynomial perhaps. :) Feb 1, 2015 at 18:58

If your data is non-negative and discrete (as in the case with number of cars on the road), you could model using a generalized linear model (GLM) with a log link function. This is known as Poisson regression and is helpful for modeling discrete non-negative counts such as the problem you described. The Poisson distribution is parameterized by a single value $\lambda$, which describes both the expected value and the variance of the distribution.
A standard trick is to estimate the logarithm of the desired quantity, then take its exponential, which is always positive. The drawback is that the error is optimized for the log, which treats differences in order of magnitude as equal. Another option is to do your regression as usual then project onto the feasible set (use the positive part of the output; $max(0, \cdot)$)