I am a trying to write a CNN from scratch in python but I am bit new to CNNs specifically the convolution layers as I am comfortable with the dense layers. I was reading Do filters have different weights for each input channel but I didn't completely understand and had a few questions. I wanted to confirm that if the input layer had 3 channels then for there to be 4 output channels you would need a total of 12 filters.

Also, can the weights(actually I am not completely sure that I know what the weights are, they are the values of the kernels right?) of a filter be a floating number as I commonly see them being described as -1, 0 ,1.

And finally, are the weights of the kernels. initialised once or rather are they changed during back propagation, as I used to have the notion that only the bias and dense layer would be changed.

Thank you so much for you time, any help is greatly appreciated.

EDIT: I have 1 more question, but I didn't feel it appropriate to make a whole new question. Basically, if I have a filter that looks like [0,1]. The stride is 2. Now if it has to run over [0,1,2,3,4]. First it will match [0,1] with [0,1] then with [2,3]. But won't it stop at [2,3] and never look at 4. How do CNNs solve this problem. Do they add an extra run of [0,1] and [3,4] but this would count 3 two times. Again thanks for the help.

  • $\begingroup$ If you've never drawn out a simple CNN, I suggest doing so. It will show you why a CNN is a fully-connected network with dropped connections (set equal to 0) and some parameters forced to be equal. Get some colored pencils, and try a 2x2 filter over a 3x3 image. $\endgroup$ – Dave Aug 14 at 19:31

Weights are learned with backpropogation, that's how model learns to identify different patterns with data. The point of a supervised model is to learn weights.

They can be floating value, they are usually represented as [[101][101][101]], because it identifies vertical line, and it's more intuitive to understand.

The filter usually do not contain info on depth, they are square matrix with depth equal to number of channels in input layer, with each filter layer spewing one output layer, so to get 4 output layers, you need to have 4 filters

Stride=2 means it will skip 2 every time, that is, it will go from 0,1 to 3,4

| improve this answer | |
  • 1
    $\begingroup$ Ok thanks, so weights can be floating point values, but are represented as integers usually to help make it more intuitive, and every filter takes into account every single input layer. Thanks so much you’be been of real help. $\endgroup$ – user68503 Feb 27 '19 at 16:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy