2
$\begingroup$

I have generated a pre-trained word2vec model using the Gensim framework (https://radimrehurek.com/gensim/auto_examples/index.html#documentation). The dataset has 507 sentiments(sentences) which are labeled as positive or negative. After performing all text processing, I used Gensim to generate the pre-trained word2Vec model. the model has 234 unique words with each vector having 300 dimension. However, I have a question.

How can I use the generated word2vec embedding vectors as input to CNN?

$\endgroup$
3
  • $\begingroup$ Have a look here: keras.io/examples/nlp/pretrained_word_embeddings $\endgroup$
    – Peter
    Nov 22, 2021 at 9:21
  • 1
    $\begingroup$ Thank you so much @Peter, this helps a lot. $\endgroup$
    – Angelica
    Nov 22, 2021 at 11:41
  • $\begingroup$ I add a full answer, maybe it helps others in the future $\endgroup$
    – Peter
    Nov 22, 2021 at 16:13

1 Answer 1

0
$\begingroup$

Keras provides a good example how to load pretrained word embeddings and train a model on it. The tricky part is to load the pretrained embeddings, but this is well explained in the code and can be adopted easily. Also note that you need to load the embeddings in the embedding layer, which must be "frozen" (should not be trainable). This can be achieved by seting trainable=False:

from tensorflow.keras.layers import Embedding

embedding_layer = Embedding(
    num_tokens,
    embedding_dim,
    embeddings_initializer=keras.initializers.Constant(embedding_matrix),
    trainable=False,
)

There are a number of useful resources provided by Keras related to natural language processing (NLP), including examples on semantic similarity (BERT), NER transformers, and sequence-to-sequence learning.

$\endgroup$
1
  • $\begingroup$ This is really helpful and more details are provided in the resources. Thanks again@Peter $\endgroup$
    – Angelica
    Nov 22, 2021 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.