0
$\begingroup$

In many research papers there are 'projection layers' related to BLSTM layers. For example, from here:

"we trained an 8-layer BLSTM encoder including 320 cells in each layer and direction, and the linear projection layer with 320 units followed by each BLSTM layer"

I can't understand what this means and how it works.

Any help on this topic would be appreciated. Thanks in advance!

$\endgroup$
1
  • $\begingroup$ Please, consider upvoting the answer if you found it useful, and marking it as correct if deemed so. Alternatively, please considering describing what the answer is lacking or why you think it is not correct, so that it can be improved. $\endgroup$ – noe Dec 24 '20 at 13:48
0
$\begingroup$

A "projection" is a simple linear/dense layer, that is, a matrix multiplication and a bias vector addition.

It is called projection because you "project" a representation of dimensionality $M$ into a representation space of dimensionality $N$.

Sometimes, especially for sequences or 2D data, these projections are implemented as a convolution of size 1, which is equivalent to the computations I described above.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer, but I have another question because it's a little bit ambiguous for me, are they used for the dimensionality reduction? $\endgroup$ – Selma_KA Dec 24 '20 at 14:19
  • $\begingroup$ A projection can be used to decrease the dimensionality of the representation, but it can as well be used to increase the dimensionality. $\endgroup$ – noe Dec 24 '20 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.