13
$\begingroup$

I am expanding my knowledge of the Keras package and I have been tooling with some of the available models. I have an NLP binary classification problem that I'm trying to solve and have been applying different models.

After working with some results and reading more and more about LSTM, it seems like this approach is far superior to anything else I've tried (across multiple datasets). I keep thinking to myself, "why/when would you not use LSTM?". The use of the additional gates, inherent to LSTM, makes perfect sense to me after having some models that suffer from vanishing gradients.

So what's the catch with LSTM? Where do they not do so well? I know there is no such thing as a "one size fits all" algorithm, so there must be a downside to LSTM.

Improve this question
$\endgroup$
1
  • $\begingroup$ Try GRU, they are like LSTM but require less memory and train faster. $\endgroup$
    – Vivek Khetan
    Commented Feb 3, 2018 at 16:58

1 Answer 1

Reset to default
13
$\begingroup$

You are right that LSTMs work very well for some problems, but some of the drawbacks are:

  • LSTMs take longer to train
  • LSTMs require more memory to train
  • LSTMs are easy to overfit
  • Dropout is much harder to implement in LSTMs
  • LSTMs are sensitive to different random weight initializations

These are in comparison to a simpler model like a 1D conv net, for example.

The first three items are because LSTMs have more parameters.

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ Agreed, and I think overfitting (aka poor generalization) is perhaps the biggest risk. Make sure you have a good strategy for doing model validation. $\endgroup$
    – tom
    Commented Feb 2, 2018 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.