# How to visualize convolutional network focus on autoencoder?

I'm trying to create some cool (and at least somewhat meaningful) visualizations of my CNN that I am using to autoencode images, as I use the encoded middle layer to do similarity comparisons.

I.e. I want a view of the input image but with some sort of "heat map" or something overlayed that shows what the CNN is "focusing on", if that makes sense. Some sort of gradient magnitude perhaps, I saw this on a different CNN project, e.g. analogous to the examples on this site - https://rajpurkar.github.io/mlx/visualizing-cnns/

How could I go about doing this for my autoencoder? Network is coded in Python with Keras, and I've reproduced the code below:

img = Input(shape=(IMG_W, IMG_H, IMG_C), name='input_img_')
network = Convolution2D(CONV_0, 3, 3, activation='elu', border_mode='same', name='conv_0_e')(img)
network = MaxPooling2D((2, 2), border_mode='same')(network)
network = Convolution2D(CONV_1, 3, 3, activation='elu', border_mode='same', name='conv_1_e')(network)
network = MaxPooling2D((2, 2), border_mode='same')(network)
network = Convolution2D(CONV_2, 3, 3, activation='elu', border_mode='same', name='conv_2_e')(network)
network = MaxPooling2D((2, 2), border_mode='same')(network)
network = Convolution2D(CONV_3, 3, 3, activation='elu', border_mode='same', name='conv_3_e')(network)
encoded = MaxPooling2D((2, 2), border_mode='same', name='encoded')(network)

network = Convolution2D(CONV_1, 3, 3, activation='elu', border_mode='same', name='conv_0_d')(encoded)
network = UpSampling2D((2, 2))(network)
network = Convolution2D(CONV_1, 3, 3, activation='elu', border_mode='same', name='conv_1_d')(network)
network = UpSampling2D((2, 2))(network)
network = Convolution2D(CONV_2, 3, 3, activation='elu', border_mode='same', name='conv_2_d')(network)
network = UpSampling2D((2, 2))(network)
network = Convolution2D(IMG_C, 3, 3, activation='elu', border_mode='same', name='conv_decoded')(network)
decoded = UpSampling2D((2, 2))(network)

model_inputs = img
model_outputs = decoded
model = Model(input=model_inputs, output=model_outputs)



This question is a little more open-ended, I don't think there's a "right answer" per se, but I'm very open to creative ideas of how I could visualize what the network is doing. Hate to put it this way, but I'm looking for something that I would show a layman and they'd be like "wow look at that AI go" lol. But seriously that's what I'm looking for here. Maybe a collaged image of feature maps? Let me know what you think, and I'd really appreciate any sample code!

You can visualise the activation maps of the various layers, to show how a deep CNN decomposes your input into more and more abstract building blocks. There is a good discussion on Cross Validated SE here. This explains the main idea and intuition as to how CNNs work and normally gets layman motivated to listen and learn more :-)

There is another great idea names saliency maps - [Simonyan et al.]. As well as reading that paper, I suggest watching Lecture 12 of CS231n (slides here). The general idea of 'how to do this' is, I believe, can really be applied to most deep neural networks.

Here is a sample from the linked paper:

The steps are something like this:

1. take a trained network architecture
2. choose the layer that you are interested in (be it the final layer or an intermediate layer
3. set all weights in layers prior to your chosen layer to zero