I understand how to build and train a neural network like shown below, as well as those low-level features/filters. I wonder what are those high-level features: how exactly do you obtain them from a trained neural network? (Are those like the "eigenfaces"?)
Note: the image is by NVIDIA, and I don't know the specifics of the classification problem here. If needed, suppose the network is trained to distinguish human from cat.
The image, and variants of it that are commonly used are for illustrative purposes only. They generally do not represent data that has been extracted from real CNNs.
The first "Low-level features" part of the diagram is possibly from a real network (I am not sure in this case, it looks more like a constructed filter, e.g. Sobel, to me). That is because it is feasible and relatively easy to interpret the first layer's filter weights directly as images, and the filters do indeed look like the components that they detect.
The "Mid-level features" and "High-level features" in your specific diagram have probably been constructed without using a neural network. They are likely to be an artists impression of what the high level features might be. They may have been sampled from real datasets, then just cropped and arranged into the image.
Caveat: I cannot find absolute evidence for the specific image being constructed for illustration only, just I suspect this to be the case.
It is possible to extract visualisations of features detected by deeper layers. The two common ways to do this are:
Dataset matching. Finding examples in the dataset which trigger a specific neuron to output a high value. This can be isolated to a crop of the original image, because you know the combined sizes of all the filters and pools that occur before layer you are interested in.
Optimising the input image. Using gradient ascent, but instead of changing the weights, make a cost function that scores the neuron you want to visualise and keep adjusting the input until
You can get more information from resources such as this article on feature visualisation.
Its about how the neural net learn inside. Usually in deep neural network you have multiple layers, the first layers will learn the low level feature then the more you approach the output layer the more the layers will learn the high level feature. Here its identifying faces task, that's why you see that in the first layers small patterns are learned (mainly edges ...) then faces components ...
How to obtain them : you should watch what makes neuron activated in each layer depending on the input. As you know each neuron will be activated (once the DNN is trained) for specific input combinations. By visualizing that you can get an idea about what exactly each layer has learned in term of high-low level features.
