0
$\begingroup$

Trainable projection layers are said to be a very powerful thing but after reading:

I don't understand how it works. So how does the trainable projection layer used in PRADO and pQRNN work?

$\endgroup$

1 Answer 1

0
$\begingroup$

The key idea in both the papers are word embeddings are replaced by trainable activations that are computed using a nueral network. The network (referred to as bottleneck layer) takes word projections as input. The parameters of this network is shared across tokens. This results in a trainable token representation. The projection itself is not trainable, but the representation derived from projection is trainable. Hope that clarifies.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.