1
$\begingroup$

In statistics, a sequence of random variables is homoscedastic if all its random variables have the same finite variance. This is also known as homogeneity of variance (Wikipedia).

Is this assumption only for linear regression models?

If not, is there any relative method is available to check for non-linear models?

$\endgroup$
3
  • 1
    $\begingroup$ What do you mean by: "...to check for non-linear models". Do you want to check linearity or homoskedasticity? $\endgroup$
    – Peter
    Apr 12 at 13:19
  • $\begingroup$ @Peter Homoskedasticity only. I am seeing examples that are for linear models (linear regression, ridge etc. ). Is it relevant to check Homoskedasticity for other models like, decision trees, NN etc . ?? $\endgroup$
    – Wickkiey
    Apr 12 at 13:35
  • $\begingroup$ please elaborate "a sequence of random variables is homoscedastic if all its random variables have the same finite variance." $\endgroup$ Apr 20 at 12:34
1
$\begingroup$

Heteroskedasticity is relevant in cases in which you calculate a standard error for the estimated coefficients. For instance for a regression model with a single independent variable this would be for the slope coefficient:

$$SE(\hat{\beta_1}) = \sigma \left(\frac{1}{\sum_i(x_i - \bar{x})^2}\right)$$

with (see this for more details)

$$ \hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2. $$

One of the OLS assumptions is the zero conditional mean assumption, which states that $E[u|X]=0$, so that errors average out to 0.

Another assumption is homoskedasticity, which means that there is no (auto)correlation in the residuals $E[u u'|X]=\sigma I$. So the covariance matrix (sometimes also called variance-covariance matrix) is:

$$ \sigma I = \sigma \left[ \begin{array}{rrrr} 1 & 0 & \cdots & 0\\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 \\ \end{array}\right] = \Omega .$$

The important thing is that all elements in the matrix (apart of the diagonal elements from top left to bottom right) are zero, which means "no correlation between residuals". Also there is a constant variance equal to $\sigma$.

In case $ \Omega \neq \sigma I$, you face heteroscedasticity and you would need to "model" $\Omega$, e.g. by "Feasible Generalized Least Squares" (FGLS) to get an okay estimate of the standard error (See Davidson/MacKinnon: "Econometric Theory and Methods", Ch. 7.4.).

Heteroskedasticity does not affect the estimated coefficients of the model ($\beta$). It does affect the standard error of the estimated coefficients and by that also the confidence interval and p-value.

You can use "robust standard errors" to mitigate heteroskedasticity and you can test heteroscedasticity, e.g. using a White test.

With NN or Random Forest you do not estimate something like a standard error of the estimated coefficient in the way described above. So Heteroskedasticity is not an issue here.

$\endgroup$
1
$\begingroup$

It also applies to other methods, i.e. not just linear regression. For example, ANOVA and T-test also depend on homogeneity of variance.

One method to check the homogeneity of variance, compatible with the one-way ANOVA, is the Barlett's test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.