I am trying to implement the SummaRuNNer architecture ( Nallapati et al).
The equation I am stuck at in question is: $$d = tanh(W_{d}\frac{1}{N_{d}}\sum_{j=1}^{N^{d}}[h^{f}_{j},h^{b}_{j}] + b)$$ Where, $N_{d} = $ Num of sentences in the document, $h^{f}_{j}$ and $h^{b}_{j}$ are the forward and backward passes of the $j^{th}$ sentence, $[]$ is the concatenation operation, $b$ is a bias term and $W_{d}$ is a learnable parameter.
My question is, what is the best way to implement this in PyTorch.
Also, I am aware that there is already a version of this particular architecture implemented in PyTorch here, however, I am not sure if their implementation of this equation is quite right.
