linear regression with "partitioned" data

Question

I am not a stats/math expert by any stretch of the imagination, but have been trying some linear regression with census data and think I have run across a fundamental problem/obstacle.

The response variable is a percent. The independent variables are percents by state. Here is an example of what I only know to refer to as "partitioned" data. One of the groups of independent variables is "lighting" and has the following variables and values for one city (for example):

electricity - 40%
kerosene - 10%
candle - 25%
generator - 15%
solar - 5%
other - 5%

When added together, those six variables = 100%. There are only six choices and a household can only have one type of "lighting". Using all six of these variables, causes my model to blow up producing coefficients that are off the charts and clearly something is wrong.

Any insight on how to handle or at least still make some use of partitioned data with linear regression? Is there another name for these types of variables?

Answer 1

It is due to the fact that the 6th predictor is a linear combination of the other 5. You can write:

electricity = 100%-kerosene-candle-generator-solar-other

Linear regression models do not handle linear combinations at all. However the solution is easy, since the 6th variable is embedded in the first 5, you can just drop a column since it is implied by the other 5.

Answer 2

I think it depends on what algorithm you are using to optimize the loss. If you are using the normal equations to solve $X^{T}X^{-1}$ exactly you'll face problems because in the case of perfectly correlated columns the inverse does not exist. On the other hand if you are using numerical methods like gradient descent you might actually be OK.