I am trying to implement the SummaRuNNer architecture ( Nallapati et al).

The equation I am stuck at in question is: $$d = tanh(W_{d}\frac{1}{N_{d}}\sum_{j=1}^{N^{d}}[h^{f}_{j},h^{b}_{j}] + b)$$ Where, $N_{d} = $ Num of sentences in the document, $h^{f}_{j}$ and $h^{b}_{j}$ are the forward and backward passes of the $j^{th}$ sentence, $[]$ is the concatenation operation, $b$ is a bias term and $W_{d}$ is a learnable parameter.

My question is, what is the best way to implement this in PyTorch.

Also, I am aware that there is already a version of this particular architecture implemented in PyTorch here, however, I am not sure if their implementation of this equation is quite right.


1 Answer 1


Check this GitHub repo for implementing SummaRuNNer with Pytorch.

The PyTorch Implementation Of SummaRuNNer


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.