I have a data set which deals with response variable in the order of 10-20. The scatter plot for such a regression appears linear, but the problem being when I predict for test cases using values very small compared to the trained sample the predicted response variables appear in negative values. Please note that the values of the data set predicted should not be negative.

Here is the 3d scatter plot of my data enter image description here

Is there a type of regression which has better predictive power which can perform such operations without such an error?

  • $\begingroup$ Regarding "I predict for test cases using values very small compared to the trained sample". Extrapolating is always a hard problem, and not specifically an issue due to absolute magnitude. If your measurements were from 50 to 300, and you tried to predict down to where they should be 1.0 (or up to where they should be 10,000), then you'd have exactly the same problem. Do you have any model separate to the data measurement, which can be used to constrain predictions? $\endgroup$ Mar 18, 2017 at 9:04
  • $\begingroup$ I do not have any more data I've used all I have. Is there a better regression or one which does this by the most accuracy in your experience ? $\endgroup$
    – user-116
    Mar 18, 2017 at 13:30
  • $\begingroup$ I'm not asking whether you have more data, but whether any model for your measurements exists beyond "must be positive"? E.g. if this represents some physical phenomenon, is there any theoretical model that the data should conform to, so that you are not just fitting statistical model, but have some basis in what you are measuring? If not, and you want a "better regression", please explain what regression model you have used - have you used linear regression based on minimising mean square error? $\endgroup$ Mar 18, 2017 at 13:36
  • $\begingroup$ Yes as a matter of fact. It is a physical phenomenon. That's why I didn't mention about the data set. It's about Gravitational waves analysis. I'll spare you the details but yes the values have to be positive. Also as far as a theoretical model is concerned we have evaluated that the normal calculations for such a data set take too long hence we are using a regression to estimate such calculations quickly. Keep in mind we don't need high accuracy but it should still be very close to the value. It can't be negative $\endgroup$
    – user-116
    Mar 18, 2017 at 13:59

1 Answer 1


OK, so you apply linear regression to create a model for your data, and when you use that model to predict new values, the output values don't satisfy a constraint (namely, being positive). I can think of only a few different things that might be going on here:

  • The new input values you are giving to the model are not within the allowable range for the problem. This is unlikely given your setup - presumably you know your situation well enough to determine what input values are allowed - but if this does happen to be the case, you need to change the way in which you obtain inputs.
  • A linear model is not accurate over the full range between your original input data and new input data. Without more domain knowledge, it's impossible to tell what sort of model would be a better substitute. If this is a situation where you can come up with some sort of theoretical prediction about what general form of model should describe the data, you should use that form. If not... try an exponential I guess?
  • A linear model is accurate, it's just not this particular one. You might have to do something like forcing the regression to include a particular point (e.g. the origin), for example. That would mean that you're not actually doing a full linear regression, but rather fitting a model with one fewer free parameter - e.g. in slope-intercept form, you're basically forcing the intercept to zero and only fitting the slope. Or the equivalent if using another point. This should be doable with some regression library that you can use. But if you use a linear model, it will predict negative values somewhere. You'll have to live with declaring those regions of parameter space to be outside the domain of the model (or possibly finding a way to make sense of negative values).

Based on the little information you've provided here, there's really not much more I can say. Maybe if you described your situation in more detail, I could offer more specific advice.

  • $\begingroup$ Thank you for your answer I will take into consideration all the points you've mentioned. Sorry for the lack of information, But the sheer explanation of the physics phenomenon for Gravitational waves is a bit overwhelming for the data science community. So I simplified the problem as much as I can $\endgroup$
    – user-116
    Mar 19, 2017 at 12:26
  • $\begingroup$ Yes, that would be reasonable except that the information you omitted is pretty important. You could post a more detailed version of this question on Physics, and it would probably be on topic there. $\endgroup$
    – David Z
    Mar 19, 2017 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.