2
$\begingroup$

I can't find a simple answer to this by Googling which leads me to think the answer is no, but I want to be sure...

In a feed forward network, all of the layers of weights get backpropogated, but what happens in a convolutional neural network on the backprop step? Is it only the feedforward part of the network (after the convolution and pooling layers) that gets backpropped? Which would mean that the convolutional layers are a form of static feature extraction...

$\endgroup$
  • $\begingroup$ I wrote you the answer, but based on your question, I think that you need to do some more reading about the general construct of neural networks as well as the back-propagation algorithm (the main idea of using the chain rule). $\endgroup$ – Mark.F Jul 30 '19 at 6:55
2
$\begingroup$

All layers of a neural network take part in the back-propagation process. This includes the convolutional layers and the pooling layers. In general, every step of the network that the input has to go thru, the back propagation goes thru as well (in reverse order).

However, not all layers contain trainable parameters. For example standard pooling layers (max-pooling, average-pooling) and standard activation layers (sigmoid, ReLU, softmax) don't have any parameters to adjust. They still take part in the back propagation, contributing their partial derivatives, but they just have no weights that can be updated.

The convolutional layers do contain weights that are updated during the process (the parameters of the filter and their bias).

Note: I assume that what you refer to as "feed forward part", are the fully-connected layers that are usually placed as the final layers of a network. In standard CNNs, all the network is a "feed forward part", (including convolutional layers) it just means that the input goes thru a sequential pipeline until becoming the final output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.