Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

Filter by
Sorted by
Tagged with
-1
votes
0answers
12 views

What word2vec representation has that make Convolutional Neural Networks perform with good results? [closed]

I'm doing a text classification task(document-level unit of analysis) and I compared the performance of word2vec(pre-trained and train over my own data) over vanilla RNN(one-hot-encoded vectors) and ...
0
votes
0answers
13 views

Skipgram model theory confusion

In the output layer of a skipgram model, there are $|\text{Context}|*|\text{Vocab}|$ values. And for each context word, the values are basically the dot product of the input word representation and ...
0
votes
0answers
7 views

Word2Vector multicontext CBOW model with Adam optimization

In cbow multiword context word2vec model there are two weights matrixes $$ W,W^{'}$$ Where $W$ is $I$ -> $H$ weight matrix, and $W^{'}$ is $H$ -> $U$ weight matrix and output is just softmax ...
0
votes
0answers
18 views

Clustering together words that appear together while down weighting words that appear too often

I was wondering if I could get some help finding a good model for the problem I have. I have a data set where each observation is a set of words that go together. So for example, it could be: ...
1
vote
0answers
20 views

Word2Vec vs. Doc2Vec Word Vectors

I am doing some analysis on document similarity and was also interested in word similarity. I know that doc2vec inherits from word2vec and by default trains using word vectors which we can access. My ...
0
votes
0answers
28 views

Return the most relevant columns for a given keyword

Suppose my database has column name and description for each column of each table. Need to design an interface where a user can enter a keyword and the interface will return the most relevance columns....
3
votes
2answers
72 views

Dot product for similarity in word to vector computation in NLP

In NLP while computing word to vector we try to maximize log(P(o|c)). Where P(o|c) is probability that o is outside word, given that c is center word. Uo is word vector for outside word Vc is word ...
2
votes
0answers
38 views

Is it acceptable to append information to word embeddings?

Let's say I have my 300 dimensional word embedding trained with Word2Vec and it contains 10,000 word vectors. I have additional data on the 10,000 words in the form of a vector (10,000x1), containing ...
1
vote
0answers
27 views

Adding extra feature to word2vec embeddings for use in classification

I have a word2vec model that will transform my text into vectors, and I need to use them to classify the text. My data is essentially a timeseries of chat messages, so I think that the message ...
3
votes
0answers
53 views

Word2Vec: Why do some dimensions of an embedding have an interpretation, and why does addition/subtraction of embedding vectors work?

I'm reading about Word2Vec from this source: http://jalammar.github.io/illustrated-word2vec/. Below is the heatmap of the embeddings for various words. In the source, it's claimed that we can get an ...
0
votes
0answers
12 views

Building simple documents search engine

I'm having my first steps in the NLP and at the moment I'm looking forward to building my own documents search engine. I've already got to know with TFIDF in practical way and I've also read about ...
0
votes
1answer
31 views

Continuous Bag Of Words (CBOW) network architecture?

Looking into word2vec like embeddings I found this exercise on PyTorch's website which prompts the reader to implement a CBOW network in PyTorch. My question is about the architecture to implement ...
0
votes
0answers
7 views

Projection layer function

I am trying to understand word-vectors and was reading this paper https://arxiv.org/pdf/1301.3781.pdf This paper proposes CBOW architecture which uses projection layer. What is projection layer? I ...
1
vote
0answers
25 views

How to test the quality of a word embedding?

I have trained a word2vec model using GenSim 4. The problem is that my corpus is quite small. How can I test the quality of the word embeddings I have obtained? Is there some standard measures to do ...
0
votes
0answers
29 views

Word2Vec vs LexVec vs GloVe

I'm on a NLP project and found a resource that has the three word representations mentioned in the question name, and I am struggling to find one place where they all are explained and compared. As I ...
1
vote
2answers
46 views

Evaluate document similarity / content-based recommender system

I'm planning on building a basic content-based recommender system with word2vec and cosine similarity. The data consists of 300k documents in varying length. How do I evaluate my model if I have no ...
2
votes
2answers
130 views

Differentiate between positive and negative clusters

I have applied k-means clustering on my dataset of Amazon Alexa reviews. ...
0
votes
1answer
21 views

Difference between Word Embedding and Text Embedding

I am working on a dataset of amazon alexa reviews and wish to cluster them in positive and negative clusters. I am using Word2Vec for vectorization so wanted to know the difference between Text ...
2
votes
2answers
83 views

How does tf.random.log_uniform_candidate_sampler work while generating negative samples?

I was trying to build a simple negative sampler for a Word2Vec model using TensorFlow by following the tutorial here. From what I understand, the ...
-1
votes
1answer
16 views

An algorithm for Automatic Tag Clustering [closed]

Out website dinf is somewhat like StackExchange: people are submitting small definitions of concepts. We would like to automatically assign those concepts into 'Topics'. The problem is that dinf by ...
2
votes
1answer
21 views

Does the output of the Sequence-to-Sequence encoder model exist in the same semantic space as the inputs (Word2vec)? [closed]

Does the output generated from the LSTM encoder module exist in the same semantic space as the original word vectors? If so, say for example we have a sentence and we pass it through the encoder to ...
1
vote
0answers
21 views

node2vec initial embedding

I am referring to the following implementation of node2vec: https://github.com/eliorc/node2vec Is there a way to have a "smart initialization" with node2vec, i.e., to start the algorithm ...
1
vote
2answers
101 views

Difference between CBOW and SKIP Gram word vectors

I have gone through several links but was not able to understand how CBOW and Skip Gram is trained from scratch? Any good link/blogs or books would be very helpful. ...
2
votes
1answer
33 views

Can we combine multiple K-Means Models as a single model?

I have a NLP problem statement where I use a Word2Vec embedding pre-trained model to convert key text to vectors and then on a set of terms run k-means clustering to get a final model for certain <...
1
vote
0answers
21 views

Is the magnitude of a word vector correlated with the frequency of the word in a text?

In order to find the similarity between words, cosine similarity seems to be the most common measure to use. In a conversation that I had about this topic, someone mentioned that words that mean more ...
1
vote
0answers
15 views

Initializing weights that are a pointwise product of multiple variables

In two-layer perceptrons that slide across words of text, such as word2vec and fastText, hidden layer heights may be a product of two random variables such as positional embeddings and word embeddings ...
1
vote
0answers
69 views

Cat2Vec implementation X = categorical and y = categorical

I am trying to convert categorical values (zipcodes) with Cat2Vec into a matrix which can be used as an input shape for categorical prediction of a target with binary values. After reading several ...
0
votes
0answers
10 views

How to find 'pre-requisite' relationships between sentences

I have two arrays. Each of them is consists of some sentences indicating some action. For example: ...
0
votes
1answer
171 views

Semantic similarity between two or more sentences

I need to determine how similar sentences (in meaning) are to one another. In order to do it, I have been considering an algorithm (cosine similarity) to determine the similarity between sentences. I ...
0
votes
1answer
18 views

How to determine semantic differences in NLP

I would need to determine the difference in meaning between the following two sentences: I am at home I am not at home I am at the office the first two sentences ...
1
vote
1answer
25 views

Construct word2vec (CBOW) training data from beginning of sentence

When constructing training data for CBOW, Mikolov et al. suggest using the word from the center of a context window. What is the "best" approach to capturing words at the beginning/end of a ...
0
votes
0answers
8 views

Size of data required to build instruction Embeddings

I am using different sizes of ARMv7 data to build instructions embedding using a conventional word2vec model which would output a numpy array of Embeddings that ...
0
votes
0answers
15 views

Distance to the Center of Vocabulary of Word Embeddings

Suppose I: Generate a set of word embeddings (using word2vec or similar) based on a large but specific corpus Compute the centroid of all the words in the set Find the word(s) with the smallest (say ...
0
votes
1answer
29 views

Is adding the embedded words of a sentence to represent the sentence a good approach?

I have a dataset of sentences in a non english language like : word1 word2 word3 word62 word5 word1 word2 Now i want to turn each variable length sentence to a fixed size vector to give it to my ...
0
votes
1answer
35 views

best approach to embed random length sequences of words as a fixed size vector without having a maximum length? [closed]

I have a dataset of sentences in a non-English language like: word1 word2 word3 word62 word5 word1 word2 and the length of each sentence is not fixed. Now, I want to represent each sentence as a ...
0
votes
1answer
26 views

Semantic networks: word2vec?

I have some doubts on how to represent the relationships between words in texts. Let’s suppose I have two sentences like these: ...
2
votes
2answers
58 views

Should I keep common stop-words when preprocessing for word embedding?

If I want to construct a word embedding by predicting a target word given context words, is it better to remove stop words or keep them? the quick brown fox jumped over the lazy dog or quick brown ...
0
votes
2answers
230 views

Error in using sklearn's GridSearchCV on Word2Vec

I am using the sklearn_api of gensim to create an estimator for a Word2vec model to pass it to sklearn's gridsearch . My code is as follows : ...
0
votes
0answers
23 views

While sampling the corpus via Skip-gram, should the resulting target-context pairs be kept unique for the training?

Should I prepare my training data for word2vec Skip-gram embedding as unique target-context word pairs discovered throughout the corpus? Or should the repeated occurrences of the same pairs be present ...
0
votes
0answers
34 views

How to compare cosine distances across two groups of words?

I am using a word2vec model based on Wikipedia corpus. I was looking for a way to quantify if two sets of words - s1= {a1, a2...}...
1
vote
1answer
54 views

Semantic network using word2vec

I have thousands of headlines and I would like to build a semantic network using word2vec, specifically google news file. My sentences look like ...
1
vote
1answer
140 views

Understanding Transfer Learning of Word Embeddings

I can't quite visualize how transfer learning of pre-trained word embeddings is useful in an NLP task( say named entity recognition ) . I'm studying Andrew NG's Sequence Models course and he seems to ...
2
votes
0answers
21 views

Skip Gram Negative Sampling with Logistics Regression

Given a training sentences as follow form document: ... lemon, a tablespoon of apricot jam a pinch ... Word apricot choose to be target word as t with windows size 2 Training sample with both ...
2
votes
1answer
43 views

word2vec: usefulness of context vectors in classification

I've been working on a NN-based classification system that accepts document vectors as input. I can't really talk about what I'm specifically training the neural net on, so i'm hoping for a more ...
1
vote
1answer
37 views

Two questions about word2vec and gensim

I've written the code below to try word2vec implementation of gensim. I've two questions: Even though I've removed stop words, the word "the" is listed as one of the most similar words of &...
1
vote
0answers
57 views

TFIDF and TFIDF weighted W2V with Multinomial Naive Bayes?

Although the tfidf vectors don't really follow Multinomial Distribution, yet MultinomialNB works fairly well, why is it so? Also would weighted tfidf w2v work the same way or should I use GaussianNB ...
0
votes
0answers
7 views

how to make CBOW model suitable for only 10 outpus?

I want to modify CBOW model to output only 10 classes. my question is how to make this model perform better even if the number of output is much less than the number of vocabulary output? should I use ...
0
votes
1answer
34 views

Keep word2vexc/fasttext model loaded in memory without using API

I have to use Fasttext model to return word embeddings. In test I was calling it through API. Since there are too many words to compute embeddings, API call seems to be expensive. I would like to use ...
1
vote
1answer
210 views

Text classification with Word2Vec on a larger corpus

I am working on a small project and I would like to use the word2vec technique as a text representation method. I need to classify patents but I have only a few of them labelled and to increase the ...
1
vote
1answer
42 views

Literature on selecting specific dimensions in a word embedding vector

I am aware that the different dimensions in the word embedding represents different information and algebraic operations can be performed between two embeddings for example. Can anyone point me to ...

1
2 3 4 5 6