-1
$\begingroup$

I am new to CNN, What I have learned so far about the filters is that when we are giving a training example to our model, our model updates the weights by gradient descent to minimize the loss function. So my question is how the weights are retained for a particular class label?

Question is vague as my knowledge is vague. It's my 4th hour to CNN.

Ex: If I am talking about MNSIT dataset with 10 labels, Lets say I am giving 1 image to my model initially. It will have a bigger loss for the forward pass. lets say now it came for the back pass and adjusted the weights for and minimized the loss function for that label. Now when a new label arrives for training, how will it update the weights for filters which have already been updated according to the previous label?

Please help

$\endgroup$
1
  • $\begingroup$ Question has a few misconceptions about weights, classes and labels. It is a bit confusing but looking at the first answer provided, I think it is possible to understand it. To the author: I think you should be a bit more patient, you probably understand what you asking after a few more classes. $\endgroup$ – Pedro Henrique Monforte Apr 1 '20 at 22:35
1
$\begingroup$

Gradient Descent minimizes the summation of costs for all data points in the training set. The weights in the network are universal (not specific to any class), and through gradient descent converge in such away that for each loss function all training data are minimized.

There are also multiple gradient descent algorithms. What you are describing here is something called Scholastic Gradient Descent. SGD takes in 1 label at a time and approximates the gradient for the whole training set based on the one label. Now this approximation is a pretty weak approximation as its a very small percentage of your training, so there will be alot noise and alot of cost fluctuation per update in each epoch. Something called mini-batch gradient descent is more commonly used and takes in n amount of labels to approximate the gradient and preform updates. This takes longer to converge because you are taking in N random labels instead of 1 to preform one update which takes more computing time. With the tradeoff of being slower, it is more accurate and the reason why weights are adjusted in such a way that minimizes the cost for all training cost functions.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.