Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm trying to perform sentiment analysis on some data using keras.I'm using embedding layer and then LSTM. I know that embedding layer decreases the sparsity of the one hot encodings of the words and its parameters are trained while back-propagation, but I don't know the mathematics of its implementation.
Thanks in advance.
Like all hidden layers in a neural network, an embedding layer can be thought of as a feature extractor with parameters that are automatically learned during training. So, like any other layer, the parameters are adjusted during training by the backpropagation algorithm.
The specific equations used to compute error gradients and update weights are likely to depend on the optimizer used to train the network. The classic optimizer is stochastic gradient descent, but more sophisticated optimizers like Adagrad, RMSProp, and ADAM are commonly used nowadays.
