# How neural networks handle large variations in the input?

I am reading the paper with the title "Classification of indoor actions through deep neural networks". And I came across this statement:

With this mechanism very complex functions can be learned combining these modules: the resulting networks are often very sensitive to minute details and insensitive to large irrelevant variations.

I have read a lot about deep neural network. However, I might have missed the logic behind why deep neural networks are insensitive to large irrelevant features.

They can handle large variations of inputs because the neurons have weights and those weights get optimized as part of learning a good model. So even though the model might take in a value that ranges from 0 to 100,000, if it isn't relevant to predicting the outcome, it will have a very small effect.

• How come it's very sensitive to minute details as well? So you're saying the weight takes care of the irrelevant variations in the inputs? Hmm, what is the model is already trained and it accepts a new input? Can you give a more intuitive explanation?
– mc8
Commented May 9, 2017 at 17:27
• The features that are important get a high weight and are therefore sensitive while the features that are not important get a low weight. If this is a multi-layer model, some of the "features" can really composites of other neurons so it gets more complex, but that's the basic idea. If the model is already trained and gets a new input that follows the same pattern as the training data, the weights are (supposed) to take care of that.
– CalZ
Commented May 9, 2017 at 17:42
• Intuitive example: you have a trained model in your head for assessing danger very quickly. If someone comes at you, you rapidly assess what they have in their hands. Assume a simple world where people can only hold bananas and knives. Holding_Banana has a weight of .000001 while Holding_Knife has a weight of 1. Tomorrow when you're walking around, it is largely irrelevant whether someone comes at you with a banana because of the weight.
– CalZ
Commented May 9, 2017 at 17:44
• Oh right, cause I wasn't thinking in terms of the features. I remembered that deep nets extracts high level features as you add more layers. So it doesn't matter what the input is directly, cause the output depends on the features right?
– mc8
Commented May 9, 2017 at 17:55
• Generally, yes. If your features have really uneven scales they could in theory cause problems. People typically normalize or rescale values to handle that, or you hope you account for it in training.
– CalZ
Commented May 9, 2017 at 17:58