what happens to the depth channels when convolved by multiple filters in a cnn (keras, tensorflow)

Question

I have a $15$-channel time series that I want to convolve using a $1$d CNN ($1\times n$ time-steps kernel). Now, let's say I want to have, as my first layer, $16$ filters. This would imply to my mind that the output would have a depth of $16 \times 15 = 240$, because each filter would be applied to each channel independently. However when I implement this in keras, (using Sequential) the filter dimensions in the summary do not reflect this. Here is a code fragment:

TIME_RANGE = 31
NUMBER_OF_CHANNELS = 15

model = Sequential()
model.add(Conv1D(input_shape = (TIME_RANGE, NUMBER_OF_CHANNELS),
                    filters=16,
                    kernel_size=7,
                    padding='valid',
                    data_format='channels_last'))
model.add(ReLU())
model.add(BatchNormalization())

model.add(Conv1D(   filters=32,
                    kernel_size=5,
                    padding='valid'))

and here is the corresponding summary output:

Layer (type)                 Output Shape              Param #

=================================================================

conv1d_1 (Conv1D)            (None, 25, 16)            1696

---

re_lu_1 (ReLU)               (None, 25, 16)            0

---

batch_normalization_1 (Batch (None, 25, 16)            64

---

conv1d_2 (Conv1D)            (None, 21, 32)            2592

as you can see output's shape along the time-wise axis decreases as expected due to the no-padding argument, from $31$ to $25$ to $21$, but the depth just reflects the number of filters-- so where have all my channels gone? At this point in the architecture I was expecting a depth of $15\times 16\times 32 = 7680$. It seems an implicit $1\times 1$ convolution is occurring somewhere, which I don't think I actually want-- I'd like to do my $1\times 1$ convolutions later on in the network. So what am I missing here?

Answer 1

Well I've figured it out. I need to use a depth-wise convolution. In tensorflow/keras this is implemented using DepthwiseConv2D. The depth_multiplier argument will create a new set of channels for every set of input channels, so 15 input channels with a depth multiple of 16 will create 240 output channels (instead of just 16).

Because I'm dealing with a 1D signal I just make the kernel height 1. There is an extra input dimension that also need to be 1.

Here is a fragment:

TIME_RANGE = 31
NUMBER_OF_CHANNELS = 15

from keras.layers import DepthwiseConv2D

model = Sequential()
model.add(DepthwiseConv2D(input_shape = (1, TIME_RANGE, NUMBER_OF_CHANNELS),                    
                    kernel_size=(1, 7),      # height 1,  width 7  (ostensibly 1D)
                    depth_multiplier = 16,
                    activation = 'elu',
                    padding = 'valid'))

Hope this helps someone else in the future.

BTW I got the right info from this useful blog page: https://eli.thegreenplace.net/2018/depthwise-separable-convolutions-for-machine-learning/

Answer 2

When you have a image dimension of HxWxC and you apply a convolutional layer with filter size F and kernel size KxK, you will be sliding F different KxKxC kernels over your image, resulting in the output dimension of the total operation being H'xW'xF which is what you are seeing.