Questions tagged [attention-mechanism]
The attention-mechanism tag has no usage guidance.
1
vote
0answers
18 views
How to train tensorflow's transformer model on my own data?
https://github.com/tensorflow/models/blob/master/official/transformer has an implementation of transformer model.
I want to train the model on my own data(consisting of two files, src.txt, and tgt.txt)...
2
votes
2answers
46 views
Keras Attention Guided CNN problem
I am working on a CNN for XRay image classification and I can't seem to be able to properly train it. I am trying to implement the following paper in Keras: https://arxiv.org/pdf/1801.09927.pdf
In ...
0
votes
1answer
33 views
Why and how BERT can learn different attentions for each head?
https://towardsdatascience.com/deconstructing-bert-distilling-6-patterns-from-100-million-parameters-b49113672f77
I read the blog above. It visualizes that different color/head has different ...
0
votes
0answers
15 views
Sub-Object Attention models
Questions first:
I need help to focus myself on the most relevant attention model papers (Attention to Attention if you will). Where should I start?
Have you heard of attention models that focus on ...
1
vote
0answers
19 views
How do I implement an attention mechanism for convolutional neural network in Keras?
I have a convolutional neural network in Keras on which I'd like to add an attention mechanism? Has anyone done this? It seems Keras doesn't have an in-built attention mechanism and the ones I've ...
1
vote
0answers
66 views
Why does Position Embeddings work?
In the papers "Convolutional Sequence to Sequence Learning" and
"Attention Is All You Need", positions embeddings are simply added to the input words embeddings to give the model a sense of the order ...
