In a neural network, Each neuron in the network represents some part of non-linear feature of the input. Ex: Like in mnist data, Consider the stem of number 9 is cut into multiple pieces and different part is represented by different neurons in the first hidden layer(Just an example from 3B1B neural networks video). My questions are:
What determines which neuron get to represent which part of the stem??
Is it possible that if we pass in the same input multiple times, each neuron can represent different part of the stem?? OR Is it that its all the magic of chain rule (i.e At the beginning, all neuron represent some trash feature and as updation of weights occur and then particular features have become synonymous to a particular neuron.) If so, How does this happen?
Thanks in advance