I have a time series dataset with 63 features and a univariate dependent variable. This is my first major time series project, so I was wondering if algorithms like ARIMA and LSTM are immune towards variables that have collinearity, similar to gradient boosted methods such as xgboost that is actually seen as a robust method. Below is the image of the correlation matrix. I have used xgboost with the dataset and have not got satisfactory results, even after removing some of the variables that are in the red category, so I'll proceed with the next step by using some time series methods. collinear pot

The image can always be zoomed to get a clearer description of features.


The fact that some variables may be correlated among each other cannot be the source of your unsactisfactory results. Think about it: let's say that I'm predicting the wheather, having as variables v1 the number of fingers in my hand v2 in my foot (highly correlated variables, right?). Removing v2 from the model may help a little bit because my model won't consider a noisy variable to predict the wheather.

If you are asking about colinearity, I would say that it's better to remove correlated variables, but it's not mandatory. If you are looking for better results in your time series predictor, you may want to understand how forecasting models, such as ARIMA, works in-depth. Often, this kind of models uses the past (with time series "lags", i.e. previous observations) to predict the future. Try to model lag variables (also referred to autoregressive components), moving averages (trends) and seasonality components. Although they can be very correlated between each other (look for ACF plots), they are also suitable variables for forecasting models.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.