1
$\begingroup$

There are a number of articles noting that transformers have significant advantages over "traditional" RNNs like LSTMs. And the industry as a whole have been moving away from LSTMs. My question is, in the domain of NLP, do LSTMs (or GRUs) have any practical benefits/advantages over transformers?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Please, consider marking the answer as correct or, alternatively, commenting on why you think it is not, or why it's not clear. $\endgroup$
    – noe
    Jan 8 at 9:11

1 Answer 1

Reset to default
1
$\begingroup$

LSTMs/GRUs have lower computational and memory requirements than transformers.

Depending on the case, using an LSTM instead of a Transformer may make sense due to those factors. For instance, using Transformers on mobile devices or embedded devices with CPU and memory limitations is not easy.

Also, in machine translation, having a Transformer encoder and an LSTM decoder may not significantly impact the resulting translation quality.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.