1
$\begingroup$

If I am training a deep neural net with input features that are physical in nature (e.g. temperature, precipitation, etc), and I want to be able to perform some kind of transfer learning where I train on multiple instances to see how they perform on a different set of inputs entirely. How do I make sure that the inputs being normalized in each instant don't conflict with one another?

For example, Mean temperature of 0 degrees with a standard deviation of 10 will be the same as mean temperature of 80 degrees with a standard deviation of 10 after normalization to a mean of 0 and a std dev of 1.

$\endgroup$
0
$\begingroup$

You should select a single preprocessing scheme and keep it constant for all experiments!

In your case, you want to scale your input features to $0$ mean and a standard deviation equal to $1$:

$$x_{scaled} = \frac{x-μ}{σ}$$

During the initial training compute the mean $μ$ and standard deviation $σ$ of each feature and store them. Then use the same values to scale the data for all subsequent experiments.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.