1
$\begingroup$

I understand that the 3 main layers for CNN are convolutional layer, ReLU layer and pooling layer.

However, I do not understand how CNN updates its weights and biases using backpropagation.

I understand that backpropagation uses partial derivatives. But I do not see how they are used in CNN.

Any reference is appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

the 3 main layers for CNN are convolutional layer, ReLU layer and pooling layer

Not necessarily in fact. Conv layers are the only layers that you absolutely need in order to implement a CNN. All the other are not strictly necessary. For example, I trained a CNN on Fashion MNIST dataset using ELU activation (you can check my Notebook here). Moreover, if the size of the input data is not too big and you have enough computing power, you can get away with Pooling layers and use only the Convolutional ones. (As computing power increasese, Pool layers will be progressively abandoned since they loose too much information).

Coming to your question:

how CNN updates its weights and biases using backpropagation. I understand that backpropagation uses partial derivatives. But I do not see how they are used in CNN.

Backpropagation works this way: it calculates how much each weight is contributing to the final Loss value (this computation is done finding the first partial derivative with respect to each weight). This trick works in the same way for "normal", Dense layers as for Convolutional layers. The only difference is that Conv layers are not fully connected, i.e. not all the nodes of one layer are connected with all the nodes of the following layer. The difference is that only the nodes that fall under each sliding filter are connected. That is a trick to process large amount of data (such as pixels).

Conv layers have a different architecture from Dense layers, but their training works exactly the same as for Dense layers.

Pool layers, on the other side, do not contain any learnable parameter. They are just mechanical operations that don't require any training.

EDIT: This is a great explanation of how backprop works.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I cannot find any notebook in your answer. $\endgroup$
    – Idonknow
    Commented Sep 30, 2019 at 7:35
  • $\begingroup$ May I know what are some equations required for backpropagate CNN? $\endgroup$
    – Idonknow
    Commented Sep 30, 2019 at 7:36
  • $\begingroup$ If possible, can you provide some references or give some outlines or equations on how backpropagation in CNN is carried out? $\endgroup$
    – Idonknow
    Commented Sep 30, 2019 at 7:40
  • $\begingroup$ Sorry I forgot the link. I edited the answer above and added a reference to a great blog post on backprop $\endgroup$
    – Leevo
    Commented Sep 30, 2019 at 7:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.