1
$\begingroup$

I have read about linear regression and interpreting OLS results i.e coefficients, t-value, p-value. But unable to find any material related to covariance matrix in linear regression.

I was reading about assumptions in linear regression, came across the term heteroscedasticity and was researching about its consequences. So while going through one of the answer on Stack Exchange about consequences of Heteroscedasticity I came across the term covariance matrix. Here's a link to that StackExchange

What is covariance matrix and how should one interpret it?

$\endgroup$

1 Answer 1

1
$\begingroup$

One of the OLS assumptions is the zero conditional mean assumption, which states that $E[u|X]=0$, so that errors average out to 0.

Another assumption is homoskedasticity, which means that there is no (auto)correlation in the residuals $E[u u'|X]=\sigma I$. So the covariance matrix (sometimes also called variance-covariance matrix) is:

$$ \sigma I = \sigma \left[ \begin{array}{rrrr} 1 & 0 & \cdots & 0\\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 \\ \end{array}\right] = \Omega .$$

The important thing is that all elements in the matrix (apart of the diagonal elements from top left to bottom right) are zero, which means "no correlation between residuals". Also there is a constant variance equal to $\sigma$.

In case $ \Omega \neq \sigma I$, you face heteroscedasticity and you would need to "model" $\Omega$, e.g. by "Feasible Generalized Least Squares" (FGLS).

Say you have a linear model $y=X\beta+u$ with $E(u^2)=exp(Z\gamma)$. Here $exp(Z\gamma)$ is an example for a scedastic function which describes the conditional variance. In order to obtain consistent estimates for $\gamma$, one need to obtain consistent estimates for $u$ in the first place. This can be done by using OLS to obtain $\hat{\beta}$ and $\hat{u}$.

Based on this, one can run an auxiliary linear regression $ \log \hat{u}^2 = Z \gamma + v $ to get an estimate for $\hat{\gamma}$. These estimates can be used to compute $\hat{\omega} = (\exp(Z\hat{\gamma}))^{1/2}$. Finally, FGLS estimates of $\beta$ are obtained by OLS with weights $\hat{\omega}$ which is called feasible weighted least squares.

See Davidson/MacKinnon: "Econometric Theory and Methods", Ch. 7.4.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.