9
$\begingroup$

Have there been any papers, or does anyone have any specific experience to know whether normalizing labels in a regression problem is likely to improve the performance of a neural network? I have labels that are in the range (0,1000) applying square loss in a ConvNet. I want to know if it might be useful to normalize these to a (0,1) range, or whether that's known to not matter.

$\endgroup$

1 Answer 1

12
$\begingroup$

Yes, you should do this. Given the initialization schemes and normalized inputs, the expected values for the outputs are 0. This means that you will not be too far off from the start, which helps convergence. If your target is 1000, your mean squared error will be huge which means your gradients will also be huge which can lead to numerical instabiliy.

$\endgroup$
1
  • 1
    $\begingroup$ I was perplexed about why I was getting NaN's on a regression model with labels in range (100, 300). Scaling my labels by dividing them by 300 did the trick. $\endgroup$ Mar 7, 2018 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.