# Parameter estimation in linear regression

Another test Q I couldn't answer - We have marks of students belonging to 3 sections - A,B,C and two genders - M & F. Which regression model will not be able to estimate all the parameters?

1 ) Marks = β0 + β1 Male + β2 Female + β3 section A + β4 section B + β5 section C

1. Marks = β0 + β1 Male + β4 section B + β5 section C

2. Marks = β0 + β1 Male + β5 section C

3. Marks = β0 + β1 Male

I couldn't understand this question at all. If anyone can, kindly explain. Thanks in advance.

The typical way to encode a category is with a 0/1 indicator about absence or presence of that category. If you do this, then model $$\#1$$ results in your model matrix having three columns (the sections) that add up to a vector of all $$1$$, which is in the model matrix as the intercept column (ditto for the male/female columns adding up to that same vector of $$1$$s). This means that the model matrix lacks full rank, so the OLS solution is not unique. It is typical to refer to the parameters as not being “identifiable” in such a situation, which is quite close to the phrasing in your test question.
What’s happening is that you can add $$c$$ to the intercept and not change the predictions is you subtract that same $$c$$ from $$\beta_3$$, $$\beta_4$$, and $$\beta_5$$ (ditto for subtracting $$c$$ from $$\beta_1$$ and $$\beta_2$$).