The embedding matrix which used in the initialization of the
Embedding layer is highly trained on a large corpus of text. The training and the data are so huge that the embedding has learnt a type of association between words.
A pretrained embedding like Word2Vec will produce vectors for words like
homework which are similar to each other in the embedding space.
Many such associations are learnt after rigorous training mostly on high-end machines and precisely calculated parameters.
Why is the
Embedding layer set to
As mentioned in the code, we have given a pretrained embedding matrix to the
Embedding layer through the
weights= argument. As the word suggests, its "pretrained" and requires no additional training.
We can enjoy the benefits of such an embedding by keeping it untrainable.
Additional training in the context of our task, may result in unusual behaviour of the Embedding layer and also distort the learned associations.
In some cases, the
Embedding layer is kept trainable.