Does Homoscedasticity applies only for linear regression models?

Question

In statistics, a sequence of random variables is homoscedastic if all its random variables have the same finite variance. This is also known as homogeneity of variance (Wikipedia).

Is this assumption only for linear regression models?

If not, is there any relative method is available to check for non-linear models?

Answer 1

Heteroskedasticity is relevant in cases in which you calculate a standard error for the estimated coefficients. For instance for a regression model with a single independent variable this would be for the slope coefficient:

$$SE(\hat{\beta_1}) = \sigma \left(\frac{1}{\sum_i(x_i - \bar{x})^2}\right)$$

with (see also here)

$$ \hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2. $$

One of the OLS assumptions is the zero conditional mean assumption, which states that $E[u|X]=0$, so that errors average out to 0.

Another assumption is homoskedasticity, which means that there is no (auto)correlation in the residuals $E[u u'|X]=\sigma I$. So the covariance matrix (sometimes also called variance-covariance matrix) is:

$$ \sigma I = \sigma \left[ \begin{array}{rrrr} 1 & 0 & \cdots & 0\\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 \\ \end{array}\right] = \Omega .$$

The important thing is that all elements in the matrix (apart of the diagonal elements from top left to bottom right) are zero, which means "no correlation between residuals". Also there is a constant variance equal to $\sigma$.

In case $ \Omega \neq \sigma I$, you face heteroscedasticity and you would need to "model" $\Omega$, e.g. by "Feasible Generalized Least Squares" (FGLS) to get an okay estimate of the standard error (See Davidson/MacKinnon: "Econometric Theory and Methods", Ch. 7.4.).

Heteroskedasticity does not affect the estimated coefficients of the model ($\beta$). It does affect the standard error of the estimated coefficients and by that also the confidence interval and p-value.

You can use "robust standard errors" to mitigate heteroskedasticity and you can test heteroscedasticity, e.g. using a White test.

With NN or Random Forest you do not estimate something like a standard error of the estimated coefficient in the way described above. So Heteroskedasticity is not an issue here.

Answer 2

It also applies to other methods, i.e. not just linear regression. For example, ANOVA and T-test also depend on homogeneity of variance.

One method to check the homogeneity of variance, compatible with the one-way ANOVA, is the Barlett's test.