The Keras library list only has 3 types of attentions - keras attention layers, which are :
- MultiHeadAttention layer
- Attention layer
- AdditiveAttention layer
However, in theory there are multiple types of attentions possible, e.g. (some of these may be synonyms):
- Bahdanau Attention
- Luong Attention
- what else?
Are other types just not practical or other types actually can be derived from existing implementation? Can someone please shed some light with examples?