4
$\begingroup$

Let's assume I have a random forest model and the residuals of the model are autocorrelated. Is this a problem?

As an example, let's assume I have two different random forest models, A and B, with a similar predictive performance. The residuals of model A are less autocorrelated than the residuals of model B. Should I prefer model A?

$\endgroup$
2
  • $\begingroup$ could you specify a bit more on residuals and auto-correlation? $\endgroup$
    – Aymuos
    Aug 31 '20 at 9:31
  • $\begingroup$ It is unclear what auto-correlation mean? Correlation is a relation between two random variables. There is only ONE random variable, Maybe your data is a time series? Then auto-correlation makes sense $\endgroup$ Sep 1 '20 at 0:22
7
+250
$\begingroup$

Yes, autocorrelation in residuals is a problem, but this is essentially because it is a clear illustration that there was more learnable information in the process you are modelling but your model missed it.

In the unlikely event that you have two equally performant models but one shows significant autocorrelation (you can test for this using the Durbin-Watson test as suggested in Noah Weber’s answer), this suggests neither model is working as well as we might hope (the autocorrelated model has failed to predict some predictable patterns and the other model is failing in some other way as its predictive power isn’t any better).

If you have two models that have different residuals but both are beating a naïve baseline, you’ve probably got models that will ensemble well.

$\endgroup$
6
$\begingroup$

Choose model A, if autocorrelation is significant

residuals="mistakes in predictions" should be completely random, i.e. follow White noise. Now if something is significantly autocorrelated it wont be truly random and the independent error model is incorrect and it wont be a robust variance estimator. Prefer model A

How to measure significant autocorrelation? Durbin–Watson test

$\endgroup$
3
$\begingroup$

If you fit a model and find a meaningful signal in the residuals, you should engineer more or better features to capture that signal.

A specific example is "Neglecting spatial autocorrelation causes underestimation of the error of sugarcane yield models" by Ferraciolli et al which found:

We showed that assuming independence when modeling yield leads to underestimating model errors and overfit …

They then changed feature selection process to reduce those errors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.