Linear Regression. Interpret weights as an effect on a prediction?

Question

In a basic linear regression, can I use the weights of each explanatory variable to describe their relative effects on the predicted value? If parameter A has a weight of 100, and parameter B has a weight of 10, can I say that parameter A has 10 times more effect on the outcome than parameter B?

An example: From a weather data set, I used two parameters, humidity and pressure, to make a prediction on temperature.

When I plot humidity vs. temperature, there's a very clear inverse relationship between the two. That is, the scatter plot tends down and to the right.

When I plot pressure vs. temperature, there's no relationship at all. The pressure in this report has very little variance, and the scatter plot is nearly vertical.

Based on those two plots, I intuit that changes in humidity have an effect on the high temperature, while changes in pressure have almost no effect.

I built a linear regression model (gradient descent, by hand, in Octave). Humidity had a weight of -.32386. Pressure had a weight of -.02219. Humidity's weight was 14.6 times larger than pressure's weight.

Based on that, can I state that parameter Humidity has nearly 15 times more effect on Temperature than parameter Pressure?

Forgive what must seem like a very amateur question. If you can give a relatively simple answer and point me toward resources where I can learn more, I'd really appreciate it. Thanks in advance.

Answer 1

In a basic linear regression, can I use the weights of each explanatory variable to describe their relative effects on the predicted value? If parameter A has a weight of 100, and parameter B has a weight of 10, can I say that parameter A has 10 times more effect on the outcome than parameter B? Blockquote

Somewhat yes. Precisely it means if A changes by one unit the outcome will change 100 times, on average (while B is frozen because it is a multiple linear regression -> you have more than one independent variable).

There are tons of examples and blogs explaining it, see for example here . Also you can analyze your regression model more by doing some statistical tests like identifying how variables are correlated using Variance Inflation Factors and estimate the statistical significance (p-value, F-test), here is a very nice intro.

An example: From a weather data set, I used two parameters, humidity and pressure, to make a prediction on temperature.

I do not see any problem with your analysis. As you said you see an inverse correlation between humidity vs. temperature in your plot as well as via your -.32386 weight. While the weight of -.02219 for pressure is nearly zero and negligible and obviously you won't see such relationship between pressure-temperature.

Answer 2

I think there is a bit of oversimplification in your analysis. Comparing the values of two parameters doesn't really make sense, as different parameters might have different dimensions, and by comparing parameters with different dimensions you are saying the equivalent to: this height of this building is two times higher than the weight of this car.

This question asks a similar thing, and the most reasonable choice to me is to do it with the scaled variables (that are dimensionless) instead of the measured variables, that have dimensions.