0
$\begingroup$

I am going through Dilated Residual Network blog post. In this, Under 2.Multi-scale Context aggregation heading, author mentioned this.

The last one is the 1×1 convolutions for mapping the number of channels to be the same as the input one. Therefore, the input and the output has the same number of channels. And it can be inserted to different kinds of convolutional neural networks.

I thought, we decide number of channels in the next layer and kernels will be initialized randomly. These kernels shape is decided by us which are 1x1 or 3x3 etc., So, what did author mean when he said, 1x1 convolutions for mapping the number of channels to be the same as the input one.When, Even if its 2x2 convolutional kernel, number of channels are not changed.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Normally a convolutional neural network will get flattened into a single column vector after the convolutions and then maybe be processed by dense layer. In this model, the convolution $1\times1$ is used as an output layer. It will have $C$ channels like every other layer, but it is not dilatated. Hence, you can use this layer as the input to other convolutional neural networks.

The kernel size will not influence the channels. Imagine an RGB image with 4 by 4 pixels. If we have a $2\times 2$ convolution with $2\times 2$ stride we will get an output of dimension $3\times 2 \times 2$ (without padding). Hence the channels do not change. If we have $K$ filters we will get $K$ times a $3\times 2 \times 2$ output. If the kernel size is $4\times 4$ with a stride of $2\times 2$ we will get a $3\times 1 \times 1$ (without padding) output for each of the $K$ filters. The kernel size only influences how large the receptive field of the convolutions are. Hence, it only influences how the layer is scaling the individual dimensions of the width and height (by using RGB images as an example).

If you flatten the output of a layer you will always reduce its dimensionality to $1$. In the dilated convolutional neural network they do not have a layer that flattens the input.

Improve this answer
$\endgroup$
8
  • $\begingroup$ i think, it does not have 3 channels. Number of channels are increased if you actually see.It increased till last layer and Its C in last layers. Am i wrong? $\endgroup$
    – rawwar
    Commented Mar 20, 2019 at 12:21
  • $\begingroup$ @InAFlash: For the basic network the layer always stays at $C$ channels. I assumed RGB images, hence $C=3$. I corrected this. You could also take any other layer of the basic network and feed it into another network. $\endgroup$
    – MachineLearner
    Commented Mar 20, 2019 at 12:24
  • $\begingroup$ if you see, for large network, its not. And my question was how kernel size is related to number of channels.Thanks $\endgroup$
    – rawwar
    Commented Mar 20, 2019 at 12:27
  • 1
    $\begingroup$ @InAFlash I reached the same conclusion. Kernel 1x1 can give C channels, 2C channels, any channels. Just like kernel 3x3, which can give C channels, 32C channels, any channels. So the statement "1x1 because the same channels" seems unjustified in the article. $\endgroup$
    – Esmailian
    Commented Mar 20, 2019 at 12:44
  • 2
    $\begingroup$ @InAFlash: The author just wants to emphasize that the last layer is not flattened to ensure that the output can be used for other networks. The author could have been more precise in explaining his/her point. $\endgroup$
    – MachineLearner
    Commented Mar 20, 2019 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.