Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

1
vote
1answer
6 views

Is it possible to use word2Vec to derive hyperonymy (hyponymy or ISA relation)?

It's easy to have hyperonymy in WordNet, e.g. to know that "tea" is a case of "beverage". Is it possible to use word2Vec in this way?
0
votes
0answers
6 views

Are the Doc2vec paper and framework explanation the same?

I am looking into doc2vec since it shows promising results in several articles. When I read the paper by Le & Mikolov, I was under the assumption that the paragraph vector was also used to predict ...
0
votes
1answer
37 views

Possible reasons for word2vec learning context words as most similar rather than words in similar contexts

I am observing my word2vec model learning context words as most similar rather than words in similar contexts. I don't understand why it (word2vec in general, not my model in particular) can behave ...
1
vote
2answers
22 views

Continous bag of words claimed to be unsupervised, how is it working?

I'm following these two lectures on CBOW and skip-gram word2vec models. The first is lec 12 and the next lec 13 of a deep learning series https://www.youtube.com/watch?v=syWB-YMYZvI https://www....
1
vote
1answer
11 views

sentiment analysis for multiple entry in one text

I must do sentiment analysis on a set of financial news from s&p500 for given entities (organization names), but the problem is that each news (rows in my dataset) may have more than one entity ...
0
votes
1answer
18 views

Document similarity matching between Doc2Vec documents

I am creating a Doc2Vec model out of hundreds of PDF documents. I have 17 documents that are part of this Doc2Vec that I want to use to check similarity with other documents in the Doc2Vec model. ...
1
vote
1answer
25 views

Updating Google News Word2vec Word Embedding?

Is it possible to update the Google News Word Embedding with a custom text dataset (text data pertaining to a particular domain) ? Google News Word2Vec - Word Embedding clearly helps us to come with ...
2
votes
1answer
15 views

Why ELMo's word embedding can represent the word better than glove?

I have read the code of ELMo: https://github.com/allenai/bilm-tf Based on my understanding, ELMo first init an word embedding matrix A for all the word and then ...
0
votes
0answers
9 views

Word2Vec how to handle trend sensitive data from a live feed

(Sorry for the wall of text) TL;DR - I wonder how to regularly retrain a model on trend sensitive data from a live feed. I have been working on developing a Machine Learning model with Word2Vec as ...
1
vote
0answers
26 views

Skip-thought models applied to phrases instead of sentences

My goal is to build a statistical model with domain specific phrase embeddings. To do this, I am doing research on how to build a model using skip-thought vectors, where instead of using sentence ...
1
vote
2answers
34 views

Multi Class Classification on large dataset with over 600 classes

I'm trying to train a text data for multi class classification which comprises of 1 Million rows. After cleaning the data, I'm using a sparse matrix of Word2Vec features (Feature size is 300) The ...
0
votes
0answers
11 views

Doc2Vec Multiple Label Vectors

I have been exploring gensim's Doc2Vec library and it produces some pretty interesting results, and I'm beginning to explore multi-label embeddings. Through Radim's tutorial I understood that the ...
0
votes
0answers
10 views

Sequence tokenization and pretrained embedding layers

Sequence tokenization and pretrained embedding initialization - say you have a unique (but not huge) corpus of texts, and you also load a pretrained embedding vector (for example GloVe-100d). What's ...
0
votes
1answer
20 views

Tagging documents for doc2vec

I am working on resume parsing script. I am trying to tag documents sentences with TaggedDocument function, provided by gensim. What I have managed for now is to divide every text into sentence, put ...
0
votes
0answers
17 views

What is the best way to use word2vec for bilingual text similarity?

I face a problem where I need to compute similarities over bilingual (English and French) texts. The "database" looks like this: ...
0
votes
0answers
17 views

Train word vectors in Keras to specific similarity (regression)

I am trying to learn word vectors from labelled words/sentence pairs. The positive examples (similar texts) are weighted [0.5,0.95]. When I train the standard shared (Siamese) model like this ...
0
votes
0answers
21 views

How do I measure trade-off relationship of word in text mining?

Are there any method to measure trade-off relationship of word's meaning in text mining? I'm using python 3.6 and use Word2vec. I conducted cosine similarity of words using Word2vec and It's working ...
0
votes
0answers
23 views

How do I gather negative samples for CBOW in word2vec?

I am trying to write the cbow part of wor2vec implementation, and I am not quite sure what would be qualify as a an appropriate negative sample needed for training. Lets say we have ...
0
votes
0answers
13 views

For word embedding models (Word2vec) is there any studies on the relationship between window size and the number of words to sample from that window?

There's an easy misconception that all the words within the window size hyperparameter are sampled. For example, for word2vec Cbow, for the following sentence 'The dish ran away with the spoon to ...
1
vote
2answers
111 views

How to implement word to word Co-occurence matrix in python

To implement co-occurence matrix in sucha a way that number of times word1 occured in context of word2 in neighbourhood of given value, lets say 5. There are 100 words and a list with 1000 sentences. ...
0
votes
1answer
32 views

Sequence models word2vec

I am working on data-set with more than 100,000 records. This is how the data looks like: ...
2
votes
1answer
39 views

Predicting topics for customer reviews based on topics mapped to n-grams?

I have a large number of unlabelled customer review data(text column) and my objective is to classify each review to a particular topic. Also I have a list of unigrams,bigrams and trigrams(not a part ...
1
vote
1answer
44 views

Creating similarity metric with Doc2Vec and additional features

I have a dataset which contains many features. Each record is company that has many features. For example... Company A: Keywords - data, big data, tableau, dashboards, etc. Industry - Information ...
1
vote
0answers
35 views

Can we use doc2vec to detect outlier documents?

I have a set of documents and I want to identify and remove the outlier documents. I am just wondering if doc2vec can be used for this task. Or are there any recently evolved, promising algorithms ...
0
votes
0answers
45 views

Recommending top answerers for a Question on Quora/Stackexchange sites

I want to make a tool that will tell me who can potentially answer a question on Quora or StackExchange. The tool will take input the text of a question, and output a sorted list of users who can ...
1
vote
1answer
63 views

How to correctly pass Word2Vec vectors as input to an LSTM

I am trying to build a text classifier using lstm which, in its first layer, has weights get by a Word2Vecmodel. In order to ...
3
votes
1answer
52 views

Are word embeddings further updated during training for document classification?

I am relatively new to the area of using word embeddings in NLP tasks. From a large corpus of documents, I train word2vec word embedding vectors and afterwards I am going to use these for document ...
0
votes
1answer
93 views

doc2vec - How does the inference step work in PV-DBOW

I am quite confused about how we generate new paragraph vectors in PV-DBOW? If I want to use the embeddings to classify some text how would I generate a vector for a new paragraph? In the original ...
1
vote
3answers
45 views

Word vectors to Sentence Vectors

How can I use the vectors of words in a sentence to get the vector of that sentence . I have used strategies like - Averaging the individual word vectors or a tf-idf weighted combination of the words ....
2
votes
1answer
348 views

How to implement LSTM using Doc2Vec vectors?

I would like to build a ANN for text classification, which has an LSTM layer, and using weights obtained via a Doc2Vec model ...
1
vote
2answers
49 views

word2vec - log in the objective softmax function

I'm reading a TensorFlow tutorial on Word2Vec models and got confused with the objective function. The base softmax function is the following: $P(w_t|h) = softmax(score(w_t, h) = \frac{exp[score(...
1
vote
1answer
22 views

What are the main distribution semantics based algorithms?

I am aware that LSI, RRI and word embeddings are distributional semantics models. However, I am not certain if the below mentioned are also distributional semantic models. Non-Negative Tensor ...
0
votes
1answer
109 views

Is skip-gram model in word2vec an expanded version of N-Gram model? skip-gram vs. skip-grams?

The skip-gram model of word2vec uses a shallow neural network to learn the word embedding with (input-word, context-word) data. When I read the tutorials for the skip-gram model there was not any ...
2
votes
2answers
57 views

Character Level Embeddings

I am working on a problem that current depends on word level embeddings created using Word2Vec. I am researching new methods to apply to this model and one was a character level embedding. I have not ...
2
votes
2answers
227 views

can I use public pretrained word2vec, and continue train it for domain specific text?

I have a set of reviews from apparel domain, about 100K reviews (2M words). And I want to train word2vec to do some cool NLP staff with it. However the size is not enough for creating adequate ...
1
vote
1answer
167 views

How to count number of word embeddings in Gensim Word2Vec model

I am trying to create a Word2Vec model of the the Pub Med Central corpus using the Gensim library and want to limit the total number of word embeddings to around 1 billion. I have searched high and ...
1
vote
0answers
28 views

Learning word embeddings using RNN

The common way of learning word embeddings is based on BOW, and Skip-gram models. Is it possible to train a RNN-based architecture like GRU or LSTM with random sentences from a large corpus to learn ...
0
votes
0answers
20 views

Adding and Normalizing extra features to Word2Vec representation

My problem is kind of similar to this question I am currently using a word2vec 100 features representation of my words. However, I want to add more features to have more similarity between synonyms ...
2
votes
1answer
168 views

Semi Supervised Learning without label propagation

I am trying to cluster some words by affinity. Using Word2Vec I obtained vector representation of every word that I can cluster with a normal unsupervised method. Of these words, though, I know the ...
0
votes
1answer
229 views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
0
votes
1answer
211 views

how to update the pre-trained word2vec model with new train data using genism

Hi I have used the genism to load the Spanish fasttext word2vec model with following code: ...
2
votes
1answer
96 views

Why does spark.ml.feautures.Word2Vec vectorize sentences instead of single words?

In the process of understanding how Word2Vec in Spark differs from gensim one, I got very confused by the example presented in the Spark docs (reference link: https://spark.apache.org/docs/2.2.0/ml-...
0
votes
0answers
19 views

Extract/detect IDs (like flight booking ids) from text

I'm looking to extract ids from the body of an email. The ids are similar to flight booking IDs. For example, in an email, I would like to obtain the booking reference (something like MNFF3RGC or MNF-...
0
votes
1answer
68 views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
0
votes
1answer
43 views

Doc2Vec network architecture

I have been familiarizing myself with Word2Vec and Doc2Vec. After reading multiple papers including the the ones by T Mikolov (the creator of Doc2Vec), I am not clear on how does the neural network ...
0
votes
0answers
25 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
1
vote
1answer
58 views

what is the reason behind the bad outputs gained by RNN, LSTM when using GloVe pretrained model in text classification?

the problem is with the results gained for accuracy and f1 afer training our model via pretrained models such as GloVe. when I apply CNN as a classifier, the result are good as follows: ...
1
vote
0answers
124 views

Training of word weights in Word Embedding and Word2Vec

I want to know how are the word weights updated for the embedding layer in Keras and for Word2Vec. Like for the normal model.add(Embedding(..)) and ...
0
votes
0answers
136 views

is Glove better for word similarity Skip-gram/CBOW?

While looking at the slides for lecture 2 of CS224d: Deep Learning for Natural Language Processing: Link to slides It is said in slide number 31, that count based methods (ex: LSA) for creating word ...
1
vote
1answer
319 views

how to input the data set in to a word2vec by keras?

I am new in using word2vec model, as a result, I do not know how I can prepare my dataset as an input for word2vec? I have searched a lot but the datasets in tutorials were in CSV format or just one ...