0
$\begingroup$

In convolution neural networks, we have a concept that inner layers learn fine features like lines and edges, while outer layers learn more complex shapes.

Do we have any such understanding for layers in RNNs (like LSTMs), something like inner layers understand grammar while outer layers understand more complete meanings of sentences assuming that we are using the LSTM for some natural language task like text summarization?

$\endgroup$
  • $\begingroup$ assuming we are using the LSTM for some natural language task like text summarization $\endgroup$ – sumit Jun 17 at 4:11
0
$\begingroup$

Its not like it just understands grammar.

In LSTMs the network tries to preserve the hidden states over time. By doing this they try to learn long-term dependencies in the language and relationships between words at variable distances.

LSTM does this by using its three famous gates.

  1. Forget gate - Tries to remember only the important features and relationships overtime.
  2. Input gate - Adds new information to old cell state at each time time step.
  3. Output gate - Produces new output by taking into account old cell state and output at each time step.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.