After study in time series analysis, I recognized RMSE and MAPE are the best evaluation metrics for used model in real time series application. But my queries are below as this is my first practice application project in time series analysis:
If I use past 1 year data for training and forecast for next 30 days but we will get actual data after 30days in real time. How could we integrate these RMSE and MAPE with time-series application?
As I heard from some data scientist if I skip first training cycle and calculate these RMSE and MAPE in next retraining cycle for past training cycle where I have actual data and forecast data, Is this approach useful ?
I concerned about calculated RMSE and MAPE could help to improve model performance in real time.
Could the evaluation metrics calculation every time on retaining useful from production point of view?
My thanks in advance