There are several approaches to generate psychedelic images, providing maximum activations for individual neirons in convolutional neural networks. For example there is a lot of them there https://app.slack.com/client/T040HKJE3/threads/thread/C04655480-1581762147.250800?cdn_fallback=2 or a bit in https://arxiv.org/abs/1311.2901 (M. Zeiler) or in https://www.youtube.com/watch?v=ghEmQSxT6tw But what can we get from them?
E.g. here
does presence of 6 fish’es pieces means there are mainly 6 different positions of fishes which the net saw on 1000 train images? Or this is rather caused by some corner effects or patter size? Like 224 source pixels/100 = 2.2 fishes horizontally.
I also don't understand well how they are obtained, so can't conjecture what aspects they can depict.
There is approach when we find parts of train/validation images which gives strongest activations. This is perfectly understandable for me.