I can't understand how the answers in this question answered the question. please help me to understand the following case:
Lets look at an output layer with 10 neurons. the label/target is also a vector of the size 10. and lets say we have only 2 samples/instances.
For the first sample we get:
[0.9, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
[1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 ]
and for the second sample:
[0.2, 0.9, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2]
[0 , 1 , 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 ]
The way I understood it, we can compute the loss for each sample:
Loss = (1/10)*sum((output_layer - label)**2)
This way , the n represents the number of neurons (10). the answers says the n is the number of samples (2). what is my mistake here?
And also, if I work with full batch and not mini batch, I would want to update the weights after the network went through all the samples, wich means(I think) I want a single loss function for all the samples. The only way I think about is to define the n as the number of neurons in the output layer times the number of samples. Is that how its done?
Loss_for_all_samples = (1/20)*sun((all_outputs - all_labels)**2)
Where all_outputs will be the sum of the output_layer of each sample and all_labels the sum of lables for each sample.