Using z-score for neural network normalization

Question

I've read many people use z-score to normalize their data for presenting to a neural net, and that all data should lie in a range (usually -1 to 1), but z-score can return results beyond those bounds.

So, I guess I'm right in saying the feature data then needs to be scaled (using min-max?), after calculating z-score?

Answer 1

One of the reasons z-scores are useful in neural nets is that they can allow gradient descent to converge faster. I don't think taking these z-scores and scaling them further to be between -1 and 1 will add any extra benefit, as you already have your features expressed on a common scale.