I'm trying to resolve a paradox. Suppose that I have a bunch of data points $\{x_i,y_i\}$ and calculate a slope and intercept, $m$ and $b$ such that $y=mx+b$. Also, I can calculate the error in $m$, $\sigma_m$ and the error in $b$, $\sigma_b$. Suppose further that the true model is a line $y=ax+c +\epsilon$ where $\epsilon$ is an error term that is normally distributed and has a standard deviation of $\sigma$. If I were to calculate the error in my calculated value as a result of the errors in my coefficients I would get $error^2 = \sigma_m^2 x^2 +\sigma_b^2$. This error increases with increasing $x$, a very strange property given that the standard deviation of y at any given point x is constant, i.e. $\sigma$. What am I missing? What is the resolution of this paradox?

Update: The standard error of the slope $\sigma_m$ is given by $\frac{\sigma^2}{\sum_i (x_i -\bar{x})^2}$ so $\sigma_m$ goes down inversely with the standard deviation of the X values. I thought that perhaps this resolved the paradox. However, x,and $\bar{x}$ are not constrained by the standard deviations. All of the x values can be within two units of each other and yet x may be a thousand, million, billion or more.

  • $\begingroup$ Can you write your definition and derivation of $error^2$? $\endgroup$ – Emre Mar 15 '16 at 8:17

You have some inherent error in $m$, say that you predict $m=m_0+a$, where $m_0$ is the true value and where you can control the size of $a$ in terms of your error bounds on $m$. Then you get:


The $ax$ term is what will cause your error to grow as you increase/decrease $x$ toward $\pm\infty$. In other words, your predicted line will ultimately be infinitely far from the true line (as can be visualized by drawing two lines with slightly different slopes).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.