0
$\begingroup$

I am trying to build a multivariate linear regression and the main goal is to understand how the various features impact the response by understanding the coefficients and their confidence intervals.

For this reason, I chose multivariate linear regression because the coefficients are intuitive to interpret and from the standard error and the degrees of freedom, I can get the 95% confidence interval of the coefficients. Therefore, I can tell what the impact of a unit increase of a predictor on the outcome is. I could use more complex models such as tree based models but, even if I can get the variable importance, it is not easy to quantify the coefficient of each variable.

My problem, though, is the data is time series data and hence it shows auto-correlation. I know using regression with ARIMA errors can handle the autocorrelation issue but I found it difficult to interpret the coefficients specially when d is non-zero in ARIMA(p, d, q). The attached image, for example, is model diagnosis for one of the models I have (I have to build 1000s of them).

How can I handle the autocorrelation issue and still get coefficients that are readily interpretable like in a multivariate linear regression? My residuals are not normally distributed and I am planning to use box-cox transform to see if it can solve that problem. But, I am not sure what I should use for the autocorrelation issue.enter image description here

Improve this question
$\endgroup$
1
  • $\begingroup$ How is this model "multivariate" ? ? ? The literal meaning is "a model with multiple outcomes". What you have is a multivariable model. $\endgroup$
    – Robert Long
    Commented May 1, 2022 at 20:54

1 Answer 1

Reset to default
0
$\begingroup$

To handle autocorrelation, you should try to "subtract it" from the series. In other words, using differentiation, de-seasonalization, and transformations you should try to subtract the impact of time on your time series data. Once you end up with a distribution that looks like white noise, then you can run your regression.

You can do it with an ARIMA model, simply add all the AR(), I(), and MA() components in the regression equation, and then ignore them when you evaluate the impact of other coefficients.

Improve this answer
$\endgroup$
5
  • $\begingroup$ is there anonline example that does that to refer to? $\endgroup$
    – Fisseha Berhane
    Commented Jul 18, 2019 at 8:02
  • $\begingroup$ Well, any course, textbook, source on time series analysis is good for this task. The first thing you always have to do before running a time series regression is to make the series stationary. That's all you need. Once you made it stationary, you just check the parameters that you're interested in. $\endgroup$
    – Leevo
    Commented Jul 18, 2019 at 8:14
  • $\begingroup$ so if use ARIMAX, which makes is stationary, and get coefficients, how do I interpret them? say I get ARIMA(1,2,1)? $\endgroup$
    – Fisseha Berhane
    Commented Jul 18, 2019 at 8:25
  • $\begingroup$ If your ARIMA(1, 2, 1) is making the series stationary, then you can safely ignore the time series components of the regression equations, you only care about the external variables that you want to check and study their coefficients and SE as you proposed above $\endgroup$
    – Leevo
    Commented Jul 18, 2019 at 9:03
  • $\begingroup$ but if d= 2, I am modeling y'' as a function of x'', not y as function of x. the coefficients I get in this case will show relationships between x'' and y' '. How do I get the coefficients with the original data? $\endgroup$
    – Fisseha Berhane
    Commented Jul 18, 2019 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.