# Multicollinearity and impact of individual features

Assume the following scenario:

1. I have four features: $$x_1$$, $$x_2$$, $$x_3$$, and $$x_4$$
2. There are non-negligible multi-collinearity among the features.
3. I want to predict $$y$$ (response variable) with those 4 features.
4. I use simple multiple linear regression model: $$y = a_1x_1 + a_2x_2 + a_3x_3 + a_4x_4$$

Let's say that I want to understand the impact that different components of a chair have on the chair's retail price. For example:

$$y\,\,\,$$ = chair's retail price

$$x_1$$ = color of cushion used

$$x_2$$ = overall design of a chair

$$x_3$$ = strength of a chair

$$x_4$$ = softness of a chair

$$x_1$$ is completely independent, but other features are all somewhat impacted by the other features due to multicollinearity. For example, changing the color of cushion changes the design of a chair. Changing the design (structure) of a chair changes the strength of a chair.

I've heard that the analysis of regression coefficients are unreliable under severe multi-collinearity.

Assuming that the multiple regression model fits the chair price well, can I naively use each feature's regression coefficient to understand the impact of each feature on the response variable? If not, what technique should I use?

Ex 1: If I use a red cushion ($$x_1$$), I can increased the retail price by 3 dollars

Ex 2: If I use conference room style chair ($$x_2$$), I can increase the retail price by 12 dollars

• how strong is the correlation between your $x$. Nov 10, 2019 at 19:44

To my best knowledge, there is no easy-to-go way to mitigate one or both of the described effects. You should carefully check the correlation between your $$x$$ and decide if multicollinearity is a problem. If this is the case, and you believe (for theoretical reasons), that all the highly correlated $$x$$ are important, you could try to find other representations of (some) $$x$$ to mitigate multicollinearity, e.g. dummy/indicator representations.
• I asked this question for self-study purpose, so I don't have an actual problem. Let's say that I have a customer who wants to quantify the impact of investing his money on feature 1 on his final profit y. There are many other features that predict the final profit y. Those features are all correlated to feature 1. Can you speculate on this? How would you quantify the impact of investing, say, $400, on feature 1 on the final profit, while it is obvious that the$400 investment will have synergy effect on the other features as well, which in chain will also impact the profit y? Nov 10, 2019 at 22:13