How to normalize the features without the knowledge of the min and max values in online learning?

I am developing an online learning platform where input features are gathered from various sensors. However, these features may have vastly different ranges. For example, displacement values may be measured in millimeters while voltage values are measured in volts. To help facilitate the training of the neural network, I would like to normalize these inputs. However, the issue is that I do not have access to their respective maximum and minimum values, so using the max-min scaling method is not feasible.

For instance, at the k-1 time step, the displacement value may be 150mm and the voltage value may be 3V, whereas at the k time step, the displacement value could be 1000mm and the voltage value could be 1V. My question is, how can I scale these differing features to a range between 0-1 to feed the neural network at every time step?

Interesting challenge. You have multiple ways to deal with it:

• Pre-define min-max values

Instead of using data to estimate them, you could provide those as constants. This would require knowledge of the data, but that would work.

You can estimate the min-max values over the last $$N$$ time steps. If your data doesn't vary widely, this should yield relatively stable min and max values. To avoid outliers, you could exclude extreme values (i.e. only care about the $$[10\%; 90\%]$$ range of your data)

• (Running) standardization

Instead of min-max scaling, you could use $$z= (x−μ)/σ$$ to subtract the mean and divide by the standard deviation. You can also use a running mean and a running standard deviation there. ​

• any Sigmoid function

There are many sigmoid functions (i.e. logistic or hyperbolic tangent) that will give you values within a particular range, typically $$[0;1]$$ or $$[-1;1]$$.

Conclusion

You have a lot of options to deal with this functionally. But you need to experiment to see which method preserves the right level of information so that your model can perform well.