I understood that it is not necessary to scale the output of a neural Network when I predict a single value via regression.
Is it necessary do normalize the Outputs of my neural Network if I have multiple outputs that vary in magnitudes between 10^-2 and 10^4? My intuition would tell me that the loss function might ignore the smaller values and only focus on the values of a bigger magnitude.
Yes, if proper weights were not introduced in cost function or target variables were not normalized, optimization process would be driven by the target variable which had the largest scale.
Therefore it's a good practice to normalize outputs if they have huge differences in terms of scale.
You generally normalise the inputs e.g. to the range [-1, 1], such that the neural network predicts within the same range.
If you are predicting multiple output of various scales, you could just retain the scaling factors used above when scaling the input, and use them to scale the predicted outputs back to the same scale as the input.
