Statement 1 is correct, statement 2 is correct, but requires elaboration, and statement 3 is incorrect for seasonal ARIMA:
The following might point you in the right direction but hopefully you'll get a few more answers with more depth in the arena of LSTM.
You mention that you have tried both algorithms and that you are simply trying to figure out which one is better, which leads me to think you may be having more trouble with the data science process and cross validation than with the specifics of the models.
Time series in general:
Time series, in general, are difficult to forecast. If they were easy to forecast then all data scientists would be wealthy, having accurately forecast the value of all of the stocks. The reality is that hedge funds, on average, do not outperform the market and that time series forecasting is typically very poor and applies only to very short durations. The main problems are that there is a lot of noise, there are many hidden influences, models are overly simplistic, influencers do not behave as we think they should, the interplay between linearity and nonlinearity is subtle and confusing, ... ad infinitum.
ARIMA
You are incorrect in your assessment that ARIMA requires stationary time series to forecast on. Non-seasonal ARIMA has three input values to help control for smoothing, stationarity, and forecasting ARIMA(p,d,q), where:
- p is the number of autoregressive terms,
- d is the number of nonseasonal differences needed for stationarity, and
- q is the number of lagged forecast errors in the prediction equation.
By contrast seasonal ARIMA has six input values ARIMA(p,d,q,P,D,Q), where:
- P is the number of seasonal autoregressive terms,
- D is the number of seasonal differences, and
- Q is the number of seasonal moving-average terms.
Subject to the qualifying statements above, I suggest playing with seasonal ARIMA to get a feel for the intricacies involved in smoothing, de-seasoning, de-trending, de-noiseing, and forecasting.
LSTM
I don't know enough about LSTM to add much here. I will add that red flags tend to be raised when someone begins at data science exercise with deep learning. I suggest learning as much as you can using ARIMA and then applying some of your ARIMA expertise to help you learn LSTM. Neural networks can be a very powerful tool, but they:
- can take a long time to run,
- often require more data to train than other models, and
- have lots of input parameters to tune.
Cross validation and comparing models:
Time series are fun in that all training data can usually be turned into supervised learning training sets. Once can simply take a time series and roll back time. That is... pick a point in time and pretend that you don't have any additional data, then produce a forecast and see how well you did. You can march through the time series doing this $n$ times in order to get an assessment of the performance of your modelmarch through the time series doing this $n$ times in order to get an assessment of the performance of your model and to compare models while taking the necessary precautions to prevent overfitting.
Hope this helps and good luck!