0
$\begingroup$

I am currently using pytorch to implement a BLSTM-based neural network. I understand that the output of the BLSTM is two times the hidden size. However, I am currently unable to find out whether this is ordered as [forward_state_0, backward_state_n, forward_state_1, backward_state_n-1,..., forward_state_n, backward_state_0] or as [forward_state_0, forward_state_1,..., forward_state_n, backward_state_n, backward_stat_n-1,...,backward_state_0] or something else. I'd like to feed a pairwise maximum of the outputs to the next layer so the most import thing for me is which are the corresponding forward and backward states.

$\endgroup$
  • $\begingroup$ It totally depends on the implementation, but most of the BiLSTMs I've seen have the states concatenated not in the time dimension but the embedding dimension. Anyway, can you point us to the implementation? $\endgroup$ – ncasas Apr 27 at 12:12
  • $\begingroup$ Sure thing, sorry: pytorch.org/docs/stable/_modules/torch/nn/modules/rnn.html#LSTM $\endgroup$ – Scipio Apr 27 at 12:32
0
$\begingroup$

Your answer is in the documentation of the code you linked in your comment:

For the unpacked case, the directions can be separated using output.view(seq_len, batch, num_directions, hidden_size), with forward and backward being direction 0 and 1 respectively. Similarly, the directions can be separated in the packed case.

So you just need yo separate the directions as specified above and then take the element-wise maximum with torch.max.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.