I have a univariate time series. where I'm trying to predict a current value of a variable based on the previous 10 values of the same variable. I tried three approaches: 1- linear regression where I use the previous 10 values as features. 2- ARIMA model 3- deep learning (LSTM) model
I computed the RMSE between the predicted and expected current values. To my surprise, the linear regression model is performing slightly better.
RMSE linear regression = 27.73 RMSE Arima= 29 RMSE LSTM= 28.7
I know that this could be a vague question, please let me know if you need any further information such as code snippets or plots.
My questions is: Is this expected ? I would have thought that LSTM would be the most powerful approach. Is it possible for a simple linear regression model to outperform an LSTM? Do you have any advice on how to improve the performance of LSTM ?