# Effect on regression coefficients by multiplying a constant to a feature

I was solving one quiz question on Coursera and I found an interesting question.

If you double the value of a given feature (i.e. a specific column of the feature matrix), what happens to the least-squares estimated coefficients for every other feature? (assume you have no other feature that depends on the doubled feature i.e. no interaction terms).

My questions are -

1. I think, the other coefficients will stay constant. If so, can someone tell me the logical explanation behind it?
2. What about the coefficient of the scaled feature ? will it be equal to m/2 assuming we doubled the feature and existing weight was m ?
3. what if interaction terms are included as well ? what will happen there?

So, collectively I can say that Regression coefficients are independent of change of origin but not of scale. will it be correct ? If we shift, the coefficients remain unchanged. However the coefficients will change when we scale?

• Consider the matrix form of the OLS estimator.
– Dave
Jan 1 at 17:48

1. Other coefficients will stay constant because scaling a feature doesn't change the correlation between the feature and the dependent variable. The intuition might be that "rests" of prediction between y~x and y~2x are the same (ax = (a/2)(2x)) and other coefficients don't have to change.
2. Yes, it will be equal to m/2 because as I mentioned (ax = (a/2)(2x)) to do the same job (still best possible prediction).
3. It depends on the type of interaction, but if the interaction is multiplying features, the coefficient should be divided by 2 (ax1x2 = (a/2)(2x1)(x2)).

Let's check it out in an experiment:

import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression

np.random.seed(1)
x1, x2 = np.random.rand(100), np.random.rand(100)
df = pd.DataFrame({'x1': x1, 'x2': x2, 'x12':x1*x2, 'y': 2*x1 + 3*x2 + np.random.normal(0, 0.1, 100)})

model = LinearRegression().fit(df[['x1', 'x2', 'x12']], df['y'])

print('Initial')
print("Coefficients:", np.round(model.coef_,3))
print("Intercept:", round(model.intercept_,3))

df['x1'] *= 2
df['x12'] = df.x1*df.x2
model = LinearRegression().fit(df[['x1', 'x2', 'x12']], df['y'])

print()
print('x1 doubled')
print("Coefficients:", np.round(model.coef_,3))
print("Intercept:", round(model.intercept_,3))


We get:

Initial
Coefficients: [ 2.03   3.032 -0.095]
Intercept: 0.002

x1 doubled
Coefficients: [ 1.015  3.032 -0.048]
Intercept: 0.002


And that is consistent with out expectations.