Connection between the first pooling layer and the second convolutional layer

Question

This is a follow-up question regarding this question. I managed to verify in python that the output of the first pooling layer will be a $14 \times 14\times 32$ tensor.

I interpret this as having 32 $14 \times 14$ images. Maybe my interpretation is mistaken. In the second convolutional layer we apply $64$ filters.

When I type in python conv2.shape I get

TensorShape([Dimension(55000), Dimension(14), Dimension(14), Dimension(64)]).

How that comes? I mean I am in interested in the outcome of the depth, which is $64.$

Answer 1

The first dimension is the batch size. From the tutorial you're linking to:

We store the training feature data (the raw pixel values for 55,000 images of hand-drawn digits)

So it looks like your code is performing each gradient update step using all 55,000 training examples. That's unorthodox, but it's fine as long as your computer can handle it. See this video for an explanation of batching in gradient descent.

Your interpretation of the shape as being 32 14x14 images is fine for now, but it is more proper to refer to these 32 matrices as "feature maps". A grayscale image is/contains a single feature map, while a color image would consist of three of them, one for each red, green, and blue (assuming RGB color space). Also, note that at this point in the network the feature maps are no longer "images" as any human would understand the word.