1
$\begingroup$

I recently moved to python for data analysis and apparently I am stuck on the basics. I am trying to regress the parameters of the following expression: z=20+x+3*y+noise, and I get the right intercept but the x and y parameters are clearly wrong. What am I doing missing? Code below:

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf 

# generate true values, and noise around them
np.random.seed(5)
x = np.arange(1, 101)
y = np.arange(1, 101)
z = 20 + x + 3* y + np.random.normal(0, 20, 100)

data = pd.DataFrame({'x':x, 'y':y, 'z': z})

lm = smf.ols(formula='z ~ x + y', data=data).fit()

# print the coefficients
lm.summary()

returns

enter image description here

where the x and y parameters are both 1.5, instead of being 1 and 3. What's wrong?

$\endgroup$
  • $\begingroup$ Welcome to the Data Science SE! Could you print your data? It looks to me that you are adding a very large noise component (standard deviation of 20). This could easily cause your relationship to be different since you only sample 100 data points. Maybe reduce your standard deviation of your normal distribution to 1 and see how that impacts the results. $\endgroup$ – Stereo Mar 12 '17 at 12:09
1
$\begingroup$

I think that you are seeing what you are seeing because the model sees the relationship of each set of points in your data frame, which are governed by the equation:

 z = 20 + x +3*y + noise 

But all the model sees is the resulting Z, NOT that equation which you know created z.

So it attempts to build a model which considers how Z was accomplished without knowing there was noise, while KNOWING that both x and y were in this equation because you explicitly told it the are in the model.

Based on this data. (at least this is what I got without a seed, when I ran your data..so it is close with probably different Z due to different noise)

x   y   z
1   1   32.824550
2   2   21.382597
3   3   80.615424
4   4   30.958157
5   5   42.192197
6   6   75.649622
7   7   29.815352
8   8   40.167267
9   9   59.752065
10  10  53.402601

Because x and y are the same for each point and because your formula has x + 3*y + noise , z is also equal to 4*x+ noise or 4*y+ noise or 2*x +2*y+ noise for each row. There are many ways to get the same change to Z with x & y in exact proportion plus some noise.

So the regressors are being assigned equal influence plus an equal share of the noise. It is the most parsimonious evaluation of x and y.

It is not what you expect knowing the equation, but it is the answer you should get. If you use a linear model which reduces variables of zero influent, you might even get a 0 or NA value for y.

To test that lm is working, simply reverse the order of y changing the relationship between x & y within that equations and you have a completely different result. I think the one you were looking to find.

y = np.arange(100,0,-1)


            coef    std err    t    P>|t|   [0.025  0.975]
Intercept   0.0439  0.000   119.009 0.000   0.043   0.045
x           1.2184  0.038   32.471  0.000   1.144   1.293
y           3.2130  0.038   85.629  0.000   3.139   3.288

You built your model right, but in this case the data set was not created in a way to test for what you hoped to see...but it was not wrong.

$\endgroup$
  • $\begingroup$ Thanks Bethany. Indeed what I was doing was dumb (x and y being the same). But your answer is so clear that I am not going to remove my question :). $\endgroup$ – famargar Mar 13 '17 at 13:07
  • 1
    $\begingroup$ Thank you, and that was not dumb at all! Even with tons of experience, seeing tcomplex relationships in things takes work and focus, which makes it easy to miss truly simple things. And it was good to have to figure it out and explain it. $\endgroup$ – bethanyP Mar 13 '17 at 13:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.