Skip to main content
Share Your Experience: Take the 2024 Developer Survey
23 votes

How to initialize a new word2vec model with pre-trained model weights?

Thank Abhishek. I've figure it out! Here are my experiments. 1). we plot a easy example: ...
Shixiang Wan's user avatar
19 votes

How can I get a measure of the semantic similarity of words?

Word2vec does not capture similarity based on antonyms and synonyms. Word2vec would give a higher similarity if the two words have the similar context. Eg The weather in California was _____ . The ...
Trideep Rath's user avatar
18 votes

Best practical algorithm for sentence similarity

Cosine Similarity for Vector Space could be you answer. Or you could calculate the eigenvector of each sentences. But the Problem is, what is similarity? "This is a tree", "This is not ...
Christian Frei's user avatar
15 votes

What is the difference between and Embedding Layer and an Autoencoder?

Actually they are 3 different things (embedding layer, word2vec, autoencoder), though they can be used to solve similar problems. (i.e. dense representation of data) Autoencoder is a type of neural ...
Viktor's user avatar
  • 850
14 votes

Word2Vec embeddings with TF-IDF

Word2Vec algorithms (Skip Gram and CBOW) treat each word equally, because their goal to compute word embeddings. The distinction becomes important when one needs to work with sentences or document ...
Maxim's user avatar
  • 900
14 votes

BERT vs Word2VEC: Is bert disambiguating the meaning of the word vector?

BERT and ELMo are recent advances in the field. However, there is a fine but major distinction between them and the typical task of word-sense disambiguation: word2vec (and similar algorithms ...
Juanvulcano's user avatar
13 votes
Accepted

doc2vec - How does the inference step work in PV-DBOW

The original paper does a lot of hand-waving on the implementation of inference step and is not clear. So your confusion is justified. I'll explain at high level below. I'm assuming only PV-DBOW model....
hssay's user avatar
  • 1,998
12 votes

Word2Vec embeddings with TF-IDF

Train a tfidfvectorizer with your corpus and use the following code: tfidf = Tfidfvectorizer () dict(zip(tfidf.get_feature_names(), tfidf.idf_))) Now you have a ...
Aayush Shrivastav's user avatar
12 votes

Word2Vec how to choose the embedding size parameter

I have checked four well-cited papers related to word embedding: 2013 Word2Vec, 2014 GloVe, 2018 BERT, and 2018 ELMo. Only GloVe has experimented on the embedding dimension for the analogy task (...
Esmailian's user avatar
  • 9,322
11 votes

Can we take of benefit of using transfer learning while training a word2vec models?

Yes, you can take benefit of pre-trained models. Most famous one being the GoogleNewsData trained model which you can find here. Pre-trained word and phrase vectors https://drive.google.com/file/d/...
Guru's user avatar
  • 316
11 votes
Accepted

Can we compare a word2vec vector with a doc2vec vector?

In paragraph vector, the vector tries to grasp the semantic meaning of all the words in the context by placing the vector itself in each and every context. Thus finally, the paragraph vector contains ...
chmodsss's user avatar
  • 1,964
11 votes
Accepted

Word2Vec how to choose the embedding size parameter

You might find this paper might be the closest thing to what you are looking for if you don't want to treat it as a regular hyperparameter: Towards Lower Bounds on Number of Dimensions for Word ...
Simon Larsson's user avatar
10 votes

How can I get a measure of the semantic similarity of words?

In Text Analytic Tools for Semantic Similarity, they developed a algorithm in order to find the similarity between 2 sentences. But if you read closely, they find the similarity of the word in a ...
Hima Varsha's user avatar
  • 2,346
10 votes
Accepted

Features of word vectors in Word2Vec

1- The number of features: In terms of neural network model it represents the number of neurons in the projection(hidden) layer. As the projection layer is built upon distributional hypothesis, ...
chmodsss's user avatar
  • 1,964
9 votes

What is the feature matrix in word2vec?

The idea behind word2vec is to represent words by a vector of real numbers of dimension d. Therefore the second matrix is the representation of those words. The i-th line of this matrix is the ...
Pierre L.'s user avatar
  • 136
9 votes

Best practical algorithm for sentence similarity

One approach you could try is averaging word vectors generated by word embedding algorithms (word2vec, glove, etc). These algorithms create a vector for each word and the cosine similarity among them ...
Dani Mesejo's user avatar
  • 2,226
9 votes

Why do we need 2 matrices for word2vec or GloVe

Might not be the answer you are seeking, but I'll still have a go: First, quick review of word2Vec, assume we are using skip gram. A typical Word2Vec train-able model consists of 1 input layer (for ...
Kari's user avatar
  • 2,736
9 votes
Accepted

Difference between Gensim word2vec and keras Embedding layer

Yep, you're right! As you know, it's difficult for machine learning models to use natural language directly, so it helps to transform words into some meaningful numeric representation. This process ...
zachdj's user avatar
  • 2,734
8 votes

Document classification using convolutional neural network

You could reduce the length of your input data by representing your documents as series of sentence vectors instead of a longer series of word vectors. Doc2vec is one way to do this (each sentence ...
Andrew's user avatar
  • 256
8 votes

Full Doc2Vec Implementation/Repdoduction in TensorFlow?

Implementations These are all full scripts using Tensorflow, but just using Tensorflow does not ensure quality. Bad WangZ's implementation looks complete from a brief glance, although it only offers ...
Nicholas Roth's user avatar
8 votes

Training Doc2Vec and Word2Vec at the same time

You need to be careful with the assumptions you make about the doc2vec implementation. Here are some useful concepts: Word2vec has two different model implementations (Skip-gram and Continuous-bag-of-...
TitoOrt's user avatar
  • 1,872
7 votes

Number of epochs in Gensim Word2Vec implementation

Increasing the number of epochs usually benefits the quality of the word representations. In experiments I have performed where the goal was to use the word embeddings as features for text ...
geompalik's user avatar
  • 411
7 votes

How to overcome training example's different lengths when working with Word Embeddings (word2vec)

Let me suggest three simple options: average the vectors (component-wise), i.e., compute the word embedding vector for each word in the text, and average them. (as suggested by others). take the (...
D.W.'s user avatar
  • 3,371
7 votes
Accepted

Weighted sum of word vectors for document similarity

Yes, your method is valid and it has been studied before it is known as Mean of Word Embeddings (MOWE) or Sum of Word Embeddings (SOWE), although your method is more a weighted mean of vectors. I ...
Dani Mesejo's user avatar
  • 2,226
7 votes

Why do we need 2 matrices for word2vec or GloVe

why we actually need two matrices (and not one) for these models. Couldn't we use the same one for U and V? In principle, you are right, we can. But we don't want to, since the increase in the ...
Esmailian's user avatar
  • 9,322
7 votes
Accepted

Accuracy and loss don't change in CNN. Is it over-fitting?

Your dataset is highly imbalanced. Your optimization process is just minimizing the loss function, and cannot do better than a model that predicts uninteresting regardless of the input, due to the ...
David Masip's user avatar
  • 6,081
7 votes

BERT vs Word2VEC: Is bert disambiguating the meaning of the word vector?

I think there are a few misconceptions in your statements. Please take into account the following BERT does not provide word-level representation. It provides sub-words embeddings and sentence ...
noe's user avatar
  • 26.9k
7 votes
Accepted

Why is averaging the vectors required in word2vec?

The reason to average the embedded vectors of the words in a paragraph or document is to obtain a single fixed-size vector that represents the whole text. Then, the document-level vector can be used ...
noe's user avatar
  • 26.9k
7 votes
Accepted

Sum vs mean of word-embeddings for sentence similarity

TL;DR You are better off averaging the vectors. Average vs sum Averaging the word vectors is a pretty known approach to get sentence level vectors. Some people may even call that "Sentence2Vec&...
Bruno Lubascher's user avatar
6 votes
Accepted

In doc2vec, how to model correctly when many documents share the same label?

I've tried to explain the logic behind labels used in Document vectors in Doc2Vec - How to label the paragraphs (gensim) To answer your questions. 1) when two documents share the same label, then ...
chmodsss's user avatar
  • 1,964

Only top scored, non community-wiki answers of a minimum length are eligible