# Considering the output of a BLSTM in pytorch, what's the order of the elements?

I am currently using pytorch to implement a BLSTM-based neural network. I understand that the output of the BLSTM is two times the hidden size. However, I am currently unable to find out whether this is ordered as [forward_state_0, backward_state_n, forward_state_1, backward_state_n-1,..., forward_state_n, backward_state_0] or as [forward_state_0, forward_state_1,..., forward_state_n, backward_state_n, backward_stat_n-1,...,backward_state_0] or something else. I'd like to feed a pairwise maximum of the outputs to the next layer so the most import thing for me is which are the corresponding forward and backward states.

• It totally depends on the implementation, but most of the BiLSTMs I've seen have the states concatenated not in the time dimension but the embedding dimension. Anyway, can you point us to the implementation? – ncasas Apr 27 at 12:12
• Sure thing, sorry: pytorch.org/docs/stable/_modules/torch/nn/modules/rnn.html#LSTM – Scipio Apr 27 at 12:32

## 1 Answer

Your answer is in the documentation of the code you linked in your comment:

For the unpacked case, the directions can be separated using output.view(seq_len, batch, num_directions, hidden_size), with forward and backward being direction 0 and 1 respectively. Similarly, the directions can be separated in the packed case.

So you just need yo separate the directions as specified above and then take the element-wise maximum with torch.max.