Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
How LSTM are able to figure out that a particular word has occurred. Like in classical algos, We have column order. But in LSTM, Since each cell receives different words, How does it know a particular word has occurred ?
When we use tfidf, We know that each column refers to a specific word. But when we use LSTM, How is this possible. Since LSTM cell receives input based on the word observed. How does LSTM know the occurrence of word since there is no index maintained
First of all LSTM does not know the word it received, not at least in the sense that you think. LSTM like all neural net cell works with numeric vector representation. Even if you would build a numeric representation for words as a numeric id unique for each word, it is still a numeric representation.
Now, the idea is that LSTM receives the numeric representation and it use that for computations. It is up to you to provide a numeric representation which makes sense for a problem. For example if you want that words with similar meaning to be considered numerically similar than they should have values which are close in the space of the vector representations. This is the role of embedding, to provide vector representation which have some meaning for your task.
The other thing is that LSTM receive all the words in sequence. Since LSTM cells have some internal state which is updated based on the sequence of words it received during training, the point is that when you give same words again for prediction it the math will behave the same as when received the same words at training. This is the only way to describe something like you say, that the LSTM cell recognize the word. So it does not recognize the word, but providing similar input will behave in a similar way. There is no such thing as identifying words.
