Can anyone help me identify what kind of Architecture is behind this Application? Is it a "simple" Classification Network? If so how are the heatmaps generated?
https://www.youtube.com/watch?v=c_giaOUzB_8
Thanks
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This technique is called saliency maps [1][2].
From the documentation of Keras-Vis:
The idea is pretty simple. We compute the gradient of output category with respect to input image. This should tell us how output category value changes with respect to a small change in input image pixels. All the positive values in the gradients tell us that a small change to that pixel will increase the output value. Hence, visualizing these gradients, which are the same shape as the image should provide some intuition of attention.
The idea behind saliency is pretty simple in hindsight. We compute the gradient of output category with respect to input image.
$\frac{∂output}{∂input}$
This should tell us how the output value changes with respect to a small change in inputs. We can use these gradients to highlight input regions that cause the most change in the output. Intuitively this should highlight salient image regions that most contribute towards the output.
