I understand that I should be scaling features between (0, 1) before feeding them into a neural network. However, what happens if future data could be larger than my current training data? For instance, if I am training a RNN on time-series data to perform demand forecasting, the product I am forecasting may be in a growth stage that will yield higher demand numbers in the future months than the past months. Is there a way to normalize data but still allow for larger values in the future?


Yes, there is. Instead of Min-Max scaling, that shrinks any distribution in the [0, 1] interval, you can Scale the variables (in statistics, they are called Z-scores). The formula is:

(x - mean(x)) / stdev(x)

This will zero-center the distribution, and scale all the variables to a standard deviation = 1.

I strongly prefer this technique to Min-Max Scaling, since the latter is too sensitive to outlier observations and generates problems unseen, out-of-scale datapoints.

Additionally, if your analysis allows for that, consider taking the log() of a data distribution that grows exponentially. That would coerce it to a more linear growth. (I'm not sure this is what you need, it's just a possibility.)

  • $\begingroup$ Thanks for the answer, @Leevo. Is there a built-in function in SciKit-Learn that does this and also the transform of this with Numpy arrays in Python? $\endgroup$ Jun 12 '19 at 14:08
  • 1
    $\begingroup$ yes, you can use sklearn's StandardScaler. Call it like this: from sklearn.preprocessing import StandardScaler. It works with numpy objects. $\endgroup$
    – Leevo
    Jun 12 '19 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.