0
$\begingroup$

I'm studying this LSTM network: https://www.kaggle.com/paoloripamonti/twitter-sentiment-analysis

model = Sequential()
model.add(embedding_layer)
model.add(Dropout(0.5))
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation='sigmoid'))

model.summary()

I understand the input part of the embedding layer. Each word get's a unique vector that represents the meaning of the word.

The drop out will deactivate neurons. So the input for the LSTM model is the vocabulary where each token is represented by a vector.

I understand the workflow in an LSTM model. But what exactly does it do with the input? Give it a score by learning? And what is the output of the lstm?

Here is the summary:

enter image description here

Improve this question
$\endgroup$
3
  • $\begingroup$ Is the question here "How does an LSTM work?" or is the question specific to this use case? $\endgroup$
    – Andy M
    Commented May 1, 2019 at 13:19
  • $\begingroup$ Specific for this case. I know how a LSTM works (the gates, nodes etc). But I wonder what happens with my dataset and what the output is of the LSTM. Is it a 2D vector for each word? $\endgroup$
    – sg_sg94
    Commented May 1, 2019 at 13:38
  • $\begingroup$ so the input are vectors that represents tokens. The output is a vector with represents the sentiment score? Or not? $\endgroup$
    – sg_sg94
    Commented May 1, 2019 at 13:45

1 Answer 1

Reset to default
1
$\begingroup$

Here it looks like the model takes in sequences of words, which are turned into embeddings, which are then put into the LSTM layer. The output of the LSTM layer is a 100 (because this is what is specified in the model) dimensional vector. This 100 dimensional vector is then put into a Dense layer outputting a 1-dimensional object, which I am assuming is some sort of prediction. Does this help? I feel like I may be missing the question.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Did did help, thanks! What i'm still not understand. That's how the model is trained. So each word/token in the corpus get's a score. But how does the model predicts a random sentence with that corpus? $\endgroup$
    – sg_sg94
    Commented May 1, 2019 at 14:17
  • $\begingroup$ I'd have to dig into the code, but it's likely training on entire comments, not just words/tokens. $\endgroup$
    – Andy M
    Commented May 1, 2019 at 14:24
  • $\begingroup$ Where does that happen in the code? thank you $\endgroup$
    – sg_sg94
    Commented May 1, 2019 at 15:24
  • $\begingroup$ the output vector of the LSTM is 100 dim. Is this the output then for every token that represents the score for that token? $\endgroup$
    – sg_sg94
    Commented May 1, 2019 at 16:18
  • $\begingroup$ If you look for x_train in the code, you'll see where the x variable is created as a sequence of indicies using keras tokenizer.texts_to_sequences and pad_sequences. The output will be for every sequence of tokens (tweets), not individual tokens. $\endgroup$
    – Andy M
    Commented May 1, 2019 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.