Dynamic Memory networks are described here . I understand what is going on for question answering task but when it comes to sequence to sequence modeling, they describe it in 4th paragraph of 2.4 answer module.

In the sequence modeling task, we wish to label each word in the original sequence. To this end, the DMN is run in the same way as above over the input words. For word t, we replace Eq. 8 with $e^i = h^{i}_{t}$ . Note that the gates for the first pass will be the same for each word, as the question is the same. This allows for speed-up in implementation by computing these gates only once. However, gates for subsequent passes will be different, as the episodes are different.

I could not understand how the global gates are computed and why would they be equal for each word for first pass. They are going to be computed as a function of question vector (same) and word vector (different for each word). This is how vague i am on what is going on here. Can somebody explain how the piece in the paragraph mentioned come together?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.