Understanding the filter function in Convolution Neural Networks

Question

I am trying to follow the following tutorial accessible with this link.

Under the 3rd Heading, "3. Visualize the Activation Maps for Each Filter", we can see the following function:

def apply_filter(img, index, filter_list, ax):
    # set the weights of the filter in the convolutional layer to filter_list[i]
    model.layers[0].set_weights([np.reshape(filter_list[i], (4,4,1,1)), np.array([0])])
    # plot the corresponding activation map
    ax.imshow(np.squeeze(model.predict(np.reshape(img, (1, img.shape[0], img.shape[1], 1)))), cmap='gray')

I understood what they are trying to do. They are applying the filters and trying to show the output after that. But, what I don't understand is the following line:

model.layers[0].set_weights([np.reshape(filter_list[i], (4,4,1,1)), np.array([0])])

What does it mean to assign weights here and also, why are they reshaping the filter which is of 4*4 to 4*4*1*1?

Answer 1

The function set_weights on a Keras layer requires the shape of the inputs to match the shape of the weights which you are replacing. You can find out which dimensions these are by calling the get_weights method on the layer in which you are interested.

In that tutorial, it would look something like the following. We only need the first element returned by get_weights(), hence the [0]. We then see the shape of it:

In [7]: model.layers[0].get_weights[0].shape
(4, 4, 1, 1)

So the person who wrote that tutorial needed to match that shape and, as it is only considering one layer of the defined model, it is able to be hard-coded in their example.

---

Here are the docstrings for the two main functions I mentioned above:

set_weights()

In [8]: l1.set_weights?

Signature: l1.set_weights(weights)
Docstring:
Sets the weights of the layer, from Numpy arrays.

# Arguments
    weights: a list of Numpy arrays. The number
        of arrays and their shape must match
        number of the dimensions of the weights
        of the layer (i.e. it should match the
        output of `get_weights`).

# Raises
    ValueError: If the provided weights list does not match the
        layer's specifications.

get_weights()

In [9]: l1.get_weights?
Signature: l1.get_weights()
Docstring:
Returns the current weights of the layer.

# Returns
    Weights values as a list of numpy arrays

Answer 2

That line of code is turning the filters from the filter_vals list into the weights corresponding to a 2D convolutional layer so you can see what it looks like to apply some traditional filters from computer vision to an image. The output you see is what that layer would produce after applying the displayed filter on an image passed through that layer.

The reshape is necessary as Keras expects a dimension for the channel (we only have one channel since it's a grayscale image) and for the batch, which is why the code adds those two dummy dimensions.

Also there's a bug in that function:

model.layers[0].set_weights([np.reshape(filter_list[i], (4,4,1,1)), np.array([0])])

should be

model.layers[0].set_weights([np.reshape(filter_list[index], (4,4,1,1)), np.array([0])])

Hope that helps!