Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

1
vote
0answers
13 views

Word2vec compact models

Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
1
vote
1answer
24 views

Document parsing modeling and approach?

I'm relatively new to data science / machine learning (yes, I know) and am experimenting with text analysis. I only want a relatively naive approach and am looking to know whether my approach is valid ...
0
votes
0answers
12 views

define sentences with messy text data

I am extracting text from various file formats: pdf, emails, word docs, text files etc. The raw data will be processed (e.g. stemmed) but it is very likely that there are no clear sentences (e.g. ...
2
votes
2answers
43 views

Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
0
votes
0answers
17 views

Normalizing Jaccard similarity scores in relation to differences in document length

The Jaccard similarity of two documents A and B can be defined as the size of their intersection (how many tokens are in both docs) divided by the size of their union (total number of tokens found in ...
0
votes
1answer
28 views

Fasttext error while loading wiki pre-trained data

I am loading the model using gensim package this way: from gensim.models import FastText model = FastText.load_fasttext_format('wiki-news-300d-1M-subword.bin') ...
1
vote
0answers
14 views

Latent feature extraction using dnn and word2vec embeddings

I recently read a journal about a tag-aware recommender system. There is a part in the paper which I do not understand. They used word2vec first and using embeddings as input to a DNN to extract ...
1
vote
1answer
27 views

Word embeddings and punctuation symbols

I have a decent understanding of word embeddings (at its core, one can think of a word being converted into a vector of, say, 100 dimensions, and each dimension given a particular value... this allows ...
0
votes
1answer
15 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
1
vote
1answer
33 views

Word embeddings for Information Retrieval - Document search?

What are good ways to find for single sentence (query) the most similiar document (text). I asked myself if word vectors (weighted average of the documents) are suitable to map a single sentence to a ...
1
vote
0answers
14 views

Doc2Vec for dataset with several text fields: concatenate or separate models?

I have a dataset with several fields: description, name, header. I want to train doc2vec out of it, so that I could use vectors for classification. So I wonder, ...
0
votes
0answers
15 views

Word vectors using Pointwise Mutual Information

I'm trying to understand the steps in this link for creating word vectors based on PMI. To summarize, we start with a known set of keywords, compute the probability of co-occurrence of keywords and ...
1
vote
0answers
36 views

Why activation function is not needed during the runtime of an Word2Vec model

In Word2Vec trainable model, there are two different weight matrix. The matrix $W$ from input-to-hidden layer and the matrix $W'$ from hidden-to-output layer. Referring to this article, I understand ...
0
votes
0answers
18 views

SpaCy - Identifying same entities in the output of NER

Let's say that I've the following entities extracted by SpaCy - International Business Machines Corp. (Company) IBM Corporation (Company) IBM (Company) We can ...
0
votes
1answer
18 views

Understanding word2vec vectors representation

I'm trying to obtain the word2vec representation of few words using gensim. At present, this is the model that I have: ...
1
vote
1answer
14 views

Is it possible to use word2Vec to derive hyperonymy (hyponymy or ISA relation)?

It's easy to have hyperonymy in WordNet, e.g. to know that "tea" is a case of "beverage". Is it possible to use word2Vec in this way?
0
votes
0answers
42 views

Are the Doc2vec paper and framework explanation the same?

I am looking into doc2vec since it shows promising results in several articles. When I read the paper by Le & Mikolov, I was under the assumption that the paragraph vector was also used to predict ...
1
vote
2answers
59 views

Possible reasons for word2vec learning context words as most similar rather than words in similar contexts

I am observing my word2vec model learning context words as most similar rather than words in similar contexts. I don't understand why it (word2vec in general, not my model in particular) can behave ...
1
vote
2answers
25 views

Continous bag of words claimed to be unsupervised, how is it working?

I'm following these two lectures on CBOW and skip-gram word2vec models. The first is lec 12 and the next lec 13 of a deep learning series https://www.youtube.com/watch?v=syWB-YMYZvI https://www....
1
vote
1answer
22 views

sentiment analysis for multiple entry in one text

I must do sentiment analysis on a set of financial news from s&p500 for given entities (organization names), but the problem is that each news (rows in my dataset) may have more than one entity ...
0
votes
1answer
44 views

Document similarity matching between Doc2Vec documents

I am creating a Doc2Vec model out of hundreds of PDF documents. I have 17 documents that are part of this Doc2Vec that I want to use to check similarity with other documents in the Doc2Vec model. ...
1
vote
1answer
105 views

Updating Google News Word2vec Word Embedding?

Is it possible to update the Google News Word Embedding with a custom text dataset (text data pertaining to a particular domain) ? Google News Word2Vec - Word Embedding clearly helps us to come with ...
3
votes
1answer
94 views

Why ELMo's word embedding can represent the word better than glove?

I have read the code of ELMo: https://github.com/allenai/bilm-tf Based on my understanding, ELMo first init an word embedding matrix A for all the word and then ...
1
vote
0answers
30 views

Skip-thought models applied to phrases instead of sentences

My goal is to build a statistical model with domain specific phrase embeddings. To do this, I am doing research on how to build a model using skip-thought vectors, where instead of using sentence ...
2
votes
2answers
125 views

Multi Class Classification on large dataset with over 600 classes

I'm trying to train a text data for multi class classification which comprises of 1 Million rows. After cleaning the data, I'm using a sparse matrix of Word2Vec features (Feature size is 300) The ...
0
votes
1answer
147 views

Tagging documents for doc2vec

I am working on resume parsing script. I am trying to tag documents sentences with TaggedDocument function, provided by gensim. What I have managed for now is to divide every text into sentence, put ...
0
votes
0answers
29 views

What is the best way to use word2vec for bilingual text similarity?

I face a problem where I need to compute similarities over bilingual (English and French) texts. The "database" looks like this: ...
1
vote
2answers
226 views

How to implement word to word Co-occurence matrix in python

To implement co-occurence matrix in sucha a way that number of times word1 occured in context of word2 in neighbourhood of given value, lets say 5. There are 100 words and a list with 1000 sentences. ...
0
votes
1answer
53 views

Sequence models word2vec

I am working on data-set with more than 100,000 records. This is how the data looks like: ...
2
votes
1answer
53 views

Predicting topics for customer reviews based on topics mapped to n-grams?

I have a large number of unlabelled customer review data(text column) and my objective is to classify each review to a particular topic. Also I have a list of unigrams,bigrams and trigrams(not a part ...
3
votes
1answer
155 views

Creating similarity metric with Doc2Vec and additional features

I have a dataset which contains many features. Each record is company that has many features. For example... Company A: Keywords - data, big data, tableau, dashboards, etc. Industry - Information ...
1
vote
0answers
51 views

Can we use doc2vec to detect outlier documents?

I have a set of documents and I want to identify and remove the outlier documents. I am just wondering if doc2vec can be used for this task. Or are there any recently evolved, promising algorithms ...
0
votes
0answers
49 views

Recommending top answerers for a Question on Quora/Stackexchange sites

I want to make a tool that will tell me who can potentially answer a question on Quora or StackExchange. The tool will take input the text of a question, and output a sorted list of users who can ...
1
vote
1answer
158 views

How to correctly pass Word2Vec vectors as input to an LSTM

I am trying to build a text classifier using lstm which, in its first layer, has weights get by a Word2Vecmodel. In order to ...
4
votes
1answer
91 views

Are word embeddings further updated during training for document classification?

I am relatively new to the area of using word embeddings in NLP tasks. From a large corpus of documents, I train word2vec word embedding vectors and afterwards I am going to use these for document ...
0
votes
1answer
334 views

doc2vec - How does the inference step work in PV-DBOW

I am quite confused about how we generate new paragraph vectors in PV-DBOW? If I want to use the embeddings to classify some text how would I generate a vector for a new paragraph? In the original ...
2
votes
3answers
116 views

Word vectors to Sentence Vectors

How can I use the vectors of words in a sentence to get the vector of that sentence . I have used strategies like - Averaging the individual word vectors or a tf-idf weighted combination of the words ....
2
votes
1answer
588 views

How to implement LSTM using Doc2Vec vectors?

I would like to build a ANN for text classification, which has an LSTM layer, and using weights obtained via a Doc2Vec model ...
1
vote
2answers
76 views

word2vec - log in the objective softmax function

I'm reading a TensorFlow tutorial on Word2Vec models and got confused with the objective function. The base softmax function is the following: $P(w_t|h) = softmax(score(w_t, h) = \frac{exp[score(...
1
vote
1answer
23 views

What are the main distribution semantics based algorithms?

I am aware that LSI, RRI and word embeddings are distributional semantics models. However, I am not certain if the below mentioned are also distributional semantic models. Non-Negative Tensor ...
0
votes
1answer
155 views

Is skip-gram model in word2vec an expanded version of N-Gram model? skip-gram vs. skip-grams?

The skip-gram model of word2vec uses a shallow neural network to learn the word embedding with (input-word, context-word) data. When I read the tutorials for the skip-gram model there was not any ...
2
votes
2answers
186 views

Character Level Embeddings

I am working on a problem that current depends on word level embeddings created using Word2Vec. I am researching new methods to apply to this model and one was a character level embedding. I have not ...
2
votes
2answers
387 views

can I use public pretrained word2vec, and continue train it for domain specific text?

I have a set of reviews from apparel domain, about 100K reviews (2M words). And I want to train word2vec to do some cool NLP staff with it. However the size is not enough for creating adequate ...
1
vote
1answer
250 views

How to count number of word embeddings in Gensim Word2Vec model

I am trying to create a Word2Vec model of the the Pub Med Central corpus using the Gensim library and want to limit the total number of word embeddings to around 1 billion. I have searched high and ...
1
vote
0answers
54 views

Learning word embeddings using RNN

The common way of learning word embeddings is based on BOW, and Skip-gram models. Is it possible to train a RNN-based architecture like GRU or LSTM with random sentences from a large corpus to learn ...
0
votes
0answers
46 views

Adding and Normalizing extra features to Word2Vec representation

My problem is kind of similar to this question I am currently using a word2vec 100 features representation of my words. However, I want to add more features to have more similarity between synonyms ...
2
votes
1answer
210 views

Semi Supervised Learning without label propagation

I am trying to cluster some words by affinity. Using Word2Vec I obtained vector representation of every word that I can cluster with a normal unsupervised method. Of these words, though, I know the ...
0
votes
1answer
578 views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
0
votes
1answer
309 views

how to update the pre-trained word2vec model with new train data using genism

Hi I have used the genism to load the Spanish fasttext word2vec model with following code: ...
2
votes
1answer
253 views

Why does spark.ml.feautures.Word2Vec vectorize sentences instead of single words?

In the process of understanding how Word2Vec in Spark differs from gensim one, I got very confused by the example presented in the Spark docs (reference link: https://spark.apache.org/docs/2.2.0/ml-...