Denormalise data in Neural Networks

Question

I wrote a Neural Networks prediction model in Python.

My data has a few inputs and two outputs. In order to make it work, I have to normalise every column on data for good prediction results.

However, I had an issue. I run several times the prediction model with the same inputs to get the average and standard deviation of it. But obviously, those are also normalised to 0-1. On the test data I knew the min and max, so I could denormalised them. On the prediction values, I cannot the min and max real value.

How do you solve this kind of problem and if you cannot, are there any other decent prediction techniques without the need of normalisation?

Answer 1

Normalise using your training data statistics. Save the values used (e.g. mean and sd per feature), treating them as part of your model. Once you have used these values to transform input, they become fixed translate/scale factors in the model.

Use the same values to normalise test data or new inputs as required. You do not need to calculate new normalisation constants for new data. In fact doing so will most likely reduce the effectiveness of your model.

The same principle applies to interpreting output values if you need to scale those into range that your model produces. Scale according to your training data.

Answer 2

Why not use the same statistics to denormalize the results? If the test data is good enough to train a neural network then why would its statistics not be good enough to denormalize the results?

However, I would suggest not using min and max as the scale factors. The min and max can be very sensitive. I suggest using robust estimates of the mean and standard deviation to normalize.