I read some attention mechanism papers, but I could not understand how it can be applied to an image (classification, detection, etc) using a CNN model. How does it affect the alignment scores and the weights? I would like to understand the application of that mechanism on images.


1 Answer 1


I would suggest you refer to the paper by Hu et al, Show, Attend and Tell: Neural Image Caption Generation with Visual Attention.

enter image description here The brief idea is that the network learns the 'areas' to focus on that are on the feature maps (the last layer of the feature detectors) which can be in return mapped back to a certain location on the image.

It is not a complex topic but requires some knowledge in RNNs and LSTMs as well that help with sequential data. Best of luck.

  • $\begingroup$ From the feature map, how the attention decide which points/feature to take into account? $\endgroup$
    – nour
    Feb 7, 2021 at 21:35
  • $\begingroup$ How the attention decide which feature is relevant or no? $\endgroup$
    – nour
    Feb 9, 2021 at 15:19
  • $\begingroup$ It mainly focus on higher weighed region. And it term of higher means higher probability. I refer you to watch this (youtube.com/watch?v=Tm5B3jdJO5Q) . $\endgroup$
    – AIFahim
    Feb 16, 2021 at 5:36
  • $\begingroup$ This seems like a precursor to ViTs. You could replace step 3 with a Transformer as well. $\endgroup$ Nov 5, 2021 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.