I've just been reading
Zeiler, M.D. and Fergus, R., 2014, September. Visualizing and understanding convolutional networks. In European Conference on Computer Vision (pp. 818-833). Springer International Publishing. (Link, my summary)
They are both about visualizing features being learned in CNNs.
Although both papers have an introduction where you can read things like:
- our understanding of [CNN features] remains limited
- While the performance of representations has been improving significantly in the past few years, their design remains eminently empirical
- In this paper, with the aim of obtaining a better understanding of representations, we develop a family of methods to investigate CNNs and other image features by means of visualizations
- there is no clear understanding of why they perform so well, or how they might be improved
- there is still little insight into the internal operation and behavior of these complex models, or how they achieve such good performance
Although I like the images, I don't see how these methods are better than simply pushing all images through the network and showing the top-$n$ images which activate the neuron of interest most. Was this evaluated? Did the authors have any insights into the features which other authors didn't have before / without those techniques?
(The Zeiler&Fergus paper at least added the occlusion sensitivity analysis which does help. However, a big part of the paper is this filter visualization by deconv-nets. And I don't see how this helps to address any of the points mentioned above)