2
$\begingroup$

There are a number of articles noting that transformers have significant advantages over "traditional" RNNs like LSTMs. And the industry as a whole have been moving away from LSTMs. My question is, in the domain of NLP, do LSTMs (or GRUs) have any practical benefits/advantages over transformers?

$\endgroup$
0

1 Answer 1

2
$\begingroup$

LSTMs/GRUs have lower computational and memory requirements than transformers.

Depending on the case, using an LSTM instead of a Transformer may make sense due to those factors. For instance, using Transformers on mobile devices or embedded devices with CPU and memory limitations is not easy.

Also, in machine translation, having a Transformer encoder and an LSTM decoder may not significantly impact the resulting translation quality.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.