I am currently writing a machine learning pipeline for my time series application. At the end of each month, I get the data gathered, normalize it ([0, 1]), retrain the ML model with the new observation only and predict future values.
Should I be reading the entire dataset each time I get a new Observation, normalize the entire dataset, create the ML model, then predict?
How I got stuck:
- Let's say I have 1 feature and at t-1 all of the values have min/max = [0, 1000]
- At t, a new observation comes in with value = 1001
- How should I normalize the new value given that the ML model has been trained with different min/max?