I'm running some experiments with NNs (actually I'm running an LSTM classifier), and I stumbled across a question I haven't found the answer so far.
What do NNs learn first? When we train a network for classifying objects, for example, the accuracy usually starts low on the training set and increases throughout the epochs. My question is, in these early epochs, which examples of my training set the network is correctly classifying? Are they the "easiest" examples or simply the examples that come first? It seems to me that they are the easiest examples, i.e., those that are easily distinguishable from each other.
I'd be happy if there's a mathematical explanation for that. Any help is appreciated!