Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

Filter by
Sorted by
Tagged with
4 votes
1 answer
93 views

Sum vs mean of word-embeddings for sentence similarity

So, say I have the following sentences ["The dog says woof", "a king leads the country", "an apple is red"] I can embed each word using an ...
user avatar
1 vote
1 answer
34 views

Why is Word2vec regarded as a neural embedding?

In the skip-gram model, the probability that a word $w$ is part of the set of context words $\{w_o^{(i)}\}$ $(i= 1:m)$ where $m$ is the context window around the central word, is given by: $$p(w_o | ...
user avatar
  • 43
0 votes
1 answer
15 views

When would you use word2vec over BERT?

I am very new to Machine Learning and I have recently been exposed to word2vec and BERT. From what I know, word2vec provides a vector representation of words, but is limited to its dictionary ...
user avatar
1 vote
1 answer
36 views

Why we need to 'train word2vec' when word2vec itself is said to be 'pretrained'?

I get really confused on why we need to 'train word2vec' when word2vec itself is said to be 'pretrained'? I searched for word2vec pretrained embedding, thinking i can get a mapping table directly ...
user avatar
  • 359
1 vote
1 answer
35 views

Do we perform text embedding before or after train-test splitting?

Do we perform text embedding before or after train-test splitting? I know that for encoding variables, usually done after the split. However, I'm not sure if that's also the case for text processing?
user avatar
  • 359
1 vote
0 answers
13 views

How to deal with words out of the vocabulary CBOW implementation

I'm studying word2vec theory, and I decided to implement the Continuous Bag of Words model from zero. I know the primary pipeline for this: Preprocess a corpus: remove stopwords, lemmatization, etc. ...
user avatar
  • 11
0 votes
0 answers
10 views

How to choose between Genism Word2Vec and Keras embedding?

I've seen this post on the difference between keras embedding and word2vec in Genism. It gives me the impression that Word2Vec in Genism is kinda pre-trained word vectors. I wish very much the ...
user avatar
  • 359
0 votes
0 answers
8 views

The output of CBOW, compared to Skipgram

From my undertanding the desired outputs from skipgram is actually the word embedding for a word, as pointed in red in the picture. But how about CBOW? Is the goal of CBOW training also aim at the ...
user avatar
  • 359
0 votes
0 answers
13 views

Is it possible to resize or compress word embeddings?

I have a 600 dimension GloVe word embedding with me, pretrained on a set of documents which suits my use case. However, I would like to reduce the number of embeddings to 200 dimensions. Retraining ...
user avatar
1 vote
1 answer
22 views

What are different ways to reduce size of word2vec vectors file?

I am working on an application with memory constraints. We are getting vectors from python Gensim models but need to transmit copies of them to react native mobile app and potentially in-browser JS. ...
user avatar
1 vote
1 answer
16 views

Recommend products based on historical queries of other users

Given the user data as in the following: ...
user avatar
0 votes
0 answers
6 views

gensim word2vec results - why non-nearby word first?

from gensim.models import Word2Vec model = Word2Vec(sentences = [['a','b'],['c','d']], window = 9999999, min_count=1) model.wv.most_similar('a', topn=10) Above ...
user avatar
1 vote
1 answer
64 views

What are the exact differences between Word Embedding and Word Vectorization?

I am learning NLP. I have tried to figure out the exact difference between Word Embedding and Word Vectorization. However, seems like some articles use these words interchangeably. But I think there ...
user avatar
  • 13
0 votes
0 answers
11 views

Tag texts using predefined keywords based on the importance

I want to tag a list of texts using predefined keywords ex: keyword1, keyword2, keyword3. I ...
user avatar
  • 101
0 votes
1 answer
24 views

when to use Word2Vec over LSTM?

I'm trying to decide on a language training model for my code. and I wanted to know what aspects and elements should I take into consideration before picking one of them? I understand that for larger ...
user avatar
  • 1
0 votes
3 answers
48 views

Does word2vec fail for window size equal to sentence size

Will word2vec fail if sentences contain only similar words, or in other words, if the window size is equal to the sentence size? I suppose this question boils down to whether word to vec considers ...
user avatar
0 votes
1 answer
49 views

How to compute sentence embedding from word2vec model?

I am new to NLP and I'm trying to perform embedding for a clustering problem. I have created the word2vec model using Python's gensim library, but I am wondering ...
user avatar
0 votes
0 answers
14 views

How different are the word embeddings trained from Skipgram and CBOW?

Since what we are interested about usually from CBOW and Skipgram are the by-product word embeddings from the networks, how does the word embeddings they produce differ? When to use which to get the ...
user avatar
  • 359
0 votes
0 answers
14 views

Can word2vec's neural network itself, and not the embedding weights, be used for word prediction?

Given the shallow neural network that was used to train e.g.: a skip-gram model, my question is: Can we actually use this network to predict probable context words? What is the output of this network? ...
user avatar
-1 votes
2 answers
249 views

How to fit Word2Vec on test data?

I am working on a Sentiment Analysis problem. I am using Gensim's Word2Vec to vectorize my data in the following way: ...
user avatar
  • 1,241
1 vote
1 answer
20 views

How could I improve my classifier of text data?

I have a dataset with three columns "message", "city" and "has_info". Here is a sample of it: ...
user avatar
  • 113
0 votes
1 answer
216 views

How to calculate the mean average of word embedding and then compare strings using sklearn.metrics.pairwise

I am totally new to this topic, that's why I am so confused or stuck in this code for a while, but I am not sure how to solve it correctly. My goal is to write a short text embedding using vector ...
user avatar
  • 1
2 votes
0 answers
18 views

Comparing the cosine similarities of the same word representations, from two separate models (vector spaces)

I am comparing the cosine similarities of word representations derived from a BERT model and also from a static Word2Vec model. I understand that the vector spaces of the two models are inherently ...
user avatar
0 votes
0 answers
62 views

Alternatives to negative sampling in word2vec

In word2vec, the natural negative log likelihood has a term of the shape $$\log \sum_{w \in V} \exp (v_w \cdot v_c')$$ where $V$ is the set of vocabulary, $v_w$ is embedding for word, and $v_c'$ is ...
user avatar
  • 59
1 vote
1 answer
21 views

Treating Word Embeddings as Multivariate Gaussian Random Variables

I want to specify some probabilistic clustering model (such as a mixture model or lda) over words, and instead of using the traditional method of representing words as an indicator vector , I want to ...
user avatar
  • 23
0 votes
0 answers
38 views

Is gensim.models.word2vec pretrained?

If you load the gensim word2vec model like this gensim.models import word2vec model = Word2Vec(my_corpus) is it pre-trained on some data already (Other than the ...
user avatar
2 votes
2 answers
117 views

How to get word embedding in CBOW?

I find clear explanations for skip-gram model. We take the output weight matrix, multiply it with the one-hot vector of the word we want to get the embedding. How does it work in case of CBOW? I know ...
user avatar
  • 438
1 vote
0 answers
14 views

GloVe dot product optimized for non-comutative data whilst the operation itself being commutative

To my current knowledge, GloVe word vectors dot product are optimized to be the w_i ⋅ w_j = log⁡(P(ⅈ|j)) The probability being computed from a cooccurance matrix. However, dot product is a commutative ...
user avatar
  • 23
1 vote
1 answer
282 views

How to use pre-trained word2vec model generated by Gensim with Convolutional neural networks (CNN)

I have generated a pre-trained word2vec model using the Gensim framework (https://radimrehurek.com/gensim/auto_examples/index.html#documentation). The dataset has 507 sentiments(sentences) which are ...
user avatar
1 vote
0 answers
12 views

Can I get some help on how to train a Word2Vec Model on a dictionary? [closed]

I doing a project, where I'm ingesting student resumes with Word2Vec, and then I need to find the best applicant for a project position. So I have a table with a column for the applicant ID and for ...
user avatar
0 votes
0 answers
9 views

In the node2vec model derivation, what does it mean for node representations to be "Symmetric in Feature Space"?

The main derivation of the probabilistic model in Node2Vec goes as follows (paper available on ArXiv: https://arxiv.org/pdf/1607.00653.pdf): We formulate feature learning in networks as a maximum ...
user avatar
  • 101
2 votes
2 answers
121 views

How to examine if a Doc2Vec model is sufficiently trained?

I started experimenting with gensim's Doc2Vec for sentiment analysis. For the training of the embedding itself, I have seen examples using a reduced learning rate with a few 10s or even a few hundred ...
user avatar
  • 121
2 votes
0 answers
70 views

Ways to cluster word senses with word embeddings

I'm trying to semantically cluster polysemous words or word with different meanings in a corpus for my class study and I want to do it by word embeddings but I have no Idea how to reach to the ...
user avatar
1 vote
1 answer
67 views

Vector elements of word2vec?

In word2vec I understand that selecting a vector size of lets say 100 would give me a word vector which has the correlation (kind of) between the word and 100 other words in corpus. My question is are ...
user avatar
2 votes
1 answer
42 views

Using word embeddings as features in classification algorithms?

I see there are ways to combine word vectors to form documents by taking averages or weighted averages. However, as a result of averaging there is a loss of information. Are there ways to retain the ...
user avatar
1 vote
1 answer
49 views

Why do we calculate the vector of a document by averaging the vectors of all the words?

I am trying to build a search engine to query a folder of documents. Tutorials online suggest that we should obtain the vector of a document by averaging the vectors of all the words, then compare ...
user avatar
  • 51
1 vote
0 answers
19 views

default estimation method of gensim's word2vec skipgram?

I am now trying to use word2vec by estimating skipgram embeddings via NCE (noise contrastive estimation) rather than conventional negative sampling method, as a recent paper did (https://asistdl....
user avatar
  • 11
0 votes
0 answers
15 views

How to handle words not in the dictionary (while finding similar words)?

I am doing a project on Semantic text analysis where my data has column Technical skills (so I have to train data to find similar words) which are words and not sentences. So I wish to find similar ...
user avatar
  • 33
3 votes
2 answers
114 views

How can word2vec or BERT be used for previously unseen words

Is there any way to modify word2vec or BERT to extend finding out embeddings for words that were not in the training data? My data is extremely domain-specific and I don't really expect pre-trained ...
user avatar
  • 37
0 votes
0 answers
33 views

How to use AraVec word embeddings to build text classification model?

I am working with ArSarcasm dataset from Hugging Face. I have cleaned the tweets from all the noise with a few preprocessing steps, tokenized the tweets and lemmatized the tokens (snippet of code can ...
user avatar
1 vote
0 answers
49 views

Plotting cosine similarities in 3d space from word embeddings

I'm working on a project in which I want to estimate biases from a large corpus of newspaper articles using word2vec. Following this and this paper, biases are calculated by constructing dimension x ...
user avatar
0 votes
0 answers
7 views

Choosing an explainable embedding and classifier when each document only have one sentence

I have dataset with corpus of 20K documents. Each document is a short 1 sentences. I need to classify each sentence in 0/1 classes as well as being able to point exactly what words are responsible for ...
user avatar
  • 111
0 votes
0 answers
16 views

Attention to get context of words

The W2V techniques define context as a window of k words around the term, and using this learn the vector representations for words in the corpus. Attention networks can help us get the important ...
user avatar
  • 37
0 votes
1 answer
16 views

Data Set and guidance for Occupations/ Roles classification problem

I am working on a project where I need to find similar roles -- for example, Software Engineer, Soft. Engineer , Software Eng ( all should be marked similar) Currently, I have tried using the Standard ...
user avatar
0 votes
0 answers
12 views

Why accuracy didn't increase while loss reached nearly zero

I am trying to build a classifier using IMDB dataset. So I used a pre-trained Word2Vec model by google with a 300D vector for single words. here is the code: ...
user avatar
0 votes
1 answer
24 views

What is the difference between sparse and dense corpra?

I didn't got the meaning or the difference between sparse and dense corpra here in this sentence "the reason is that Skip-gram works better over sparse corpora like Twitter and NIPS, while CBOW ...
user avatar
  • 1
0 votes
0 answers
21 views

Weird entries in GloVe embeddings causing error

I'm trying to load GloVe embedding data, and when just printing out the words and their corresponding embeddings I get an anomaly. With the following code: ...
user avatar
  • 133
0 votes
1 answer
31 views

Getting context-word pairs for a continuous bag of words model and other confusions

Suppose I have a corpus with documents: ...
user avatar
  • 133
0 votes
0 answers
36 views

Genesis most_similar find synonym only (not antonyms)

Is there a way to let model.wv.most_similar in gensim return positive-meaning words only (i.e. that shows synonyms but not antonyms)? For example, if I do: ...
user avatar
0 votes
1 answer
36 views

Question regarding training data in word2vec - skip-gram

I have a very simple question regarding the training data in word2vec. In the skip-gram implementation, the training data (if I understand it correctly) is generated as pairs of words like it's shown ...
user avatar
  • 101

1
2 3 4 5
7