Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

0
votes
1answer
18 views

Sequence models word2vec

I am working on data-set with more than 100,000 records. This is how the data looks like: ...
0
votes
0answers
18 views

Classify words using other than tfidf

I'm working on email signature blocks and am trying to classify groups of words into "jobs", "countries", "first names" and others. Besides using certain characteristics of my own, I have thought of ...
1
vote
1answer
24 views

Predicting topics for customer reviews based on topics mapped to n-grams?

I have a large number of unlabelled customer review data(text column) and my objective is to classify each review to a particular topic. Also I have a list of unigrams,bigrams and trigrams(not a part ...
1
vote
1answer
19 views

Creating similarity metric with Doc2Vec and additional features

I have a dataset which contains many features. Each record is company that has many features. For example... Company A: Keywords - data, big data, tableau, dashboards, etc. Industry - Information ...
1
vote
0answers
21 views

Can we use doc2vec to detect outlier documents?

I have a set of documents and I want to identify and remove the outlier documents. I am just wondering if doc2vec can be used for this task. Or are there any recently evolved, promising algorithms ...
0
votes
0answers
41 views

Recommending top answerers for a Question on Quora/Stackexchange sites

I want to make a tool that will tell me who can potentially answer a question on Quora or StackExchange. The tool will take input the text of a question, and output a sorted list of users who can ...
1
vote
1answer
33 views

How to correctly pass Word2Vec vectors as input to an LSTM

I am trying to build a text classifier using lstm which, in its first layer, has weights get by a Word2Vecmodel. In order to ...
2
votes
1answer
41 views

Are word embeddings further updated during training for document classification?

I am relatively new to the area of using word embeddings in NLP tasks. From a large corpus of documents, I train word2vec word embedding vectors and afterwards I am going to use these for document ...
0
votes
1answer
30 views

doc2vec - How does the inference step work in PV-DBOW

I am quite confused about how we generate new paragraph vectors in PV-DBOW? If I want to use the embeddings to classify some text how would I generate a vector for a new paragraph? In the original ...
1
vote
3answers
33 views

Word vectors to Sentence Vectors

How can I use the vectors of words in a sentence to get the vector of that sentence . I have used strategies like - Averaging the individual word vectors or a tf-idf weighted combination of the words ....
2
votes
1answer
100 views

How to implement LSTM using Doc2Vec vectors?

I would like to build a ANN for text classification, which has an LSTM layer, and using weights obtained via a Doc2Vec model ...
1
vote
2answers
32 views

word2vec - log in the objective softmax function

I'm reading a TensorFlow tutorial on Word2Vec models and got confused with the objective function. The base softmax function is the following: $P(w_t|h) = softmax(score(w_t, h) = \frac{exp[score(...
1
vote
1answer
21 views

What are the main distribution semantics based algorithms?

I am aware that LSI, RRI and word embeddings are distributional semantics models. However, I am not certain if the below mentioned are also distributional semantic models. Non-Negative Tensor ...
0
votes
1answer
34 views

Is skip-gram model in word2vec an expanded version of N-Gram model? skip-gram vs. skip-grams?

The skip-gram model of word2vec uses a shallow neural network to learn the word embedding with (input-word, context-word) data. When I read the tutorials for the skip-gram model there was not any ...
2
votes
2answers
34 views

Character Level Embeddings

I am working on a problem that current depends on word level embeddings created using Word2Vec. I am researching new methods to apply to this model and one was a character level embedding. I have not ...
2
votes
2answers
78 views

can I use public pretrained word2vec, and continue train it for domain specific text?

I have a set of reviews from apparel domain, about 100K reviews (2M words). And I want to train word2vec to do some cool NLP staff with it. However the size is not enough for creating adequate ...
1
vote
1answer
68 views

How to count number of word embeddings in Gensim Word2Vec model

I am trying to create a Word2Vec model of the the Pub Med Central corpus using the Gensim library and want to limit the total number of word embeddings to around 1 billion. I have searched high and ...
1
vote
0answers
16 views

Learning word embeddings using RNN

The common way of learning word embeddings is based on BOW, and Skip-gram models. Is it possible to train a RNN-based architecture like GRU or LSTM with random sentences from a large corpus to learn ...
0
votes
0answers
17 views

Adding and Normalizing extra features to Word2Vec representation

My problem is kind of similar to this question I am currently using a word2vec 100 features representation of my words. However, I want to add more features to have more similarity between synonyms ...
2
votes
1answer
104 views

Semi Supervised Learning without label propagation

I am trying to cluster some words by affinity. Using Word2Vec I obtained vector representation of every word that I can cluster with a normal unsupervised method. Of these words, though, I know the ...
0
votes
1answer
104 views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
0
votes
1answer
104 views

how to update the pre-trained word2vec model with new train data using genism

Hi I have used the genism to load the Spanish fasttext word2vec model with following code: ...
1
vote
1answer
36 views

Why does spark.ml.feautures.Word2Vec vectorize sentences instead of single words?

In the process of understanding how Word2Vec in Spark differs from gensim one, I got very confused by the example presented in the Spark docs (reference link: https://spark.apache.org/docs/2.2.0/ml-...
0
votes
0answers
17 views

Extract/detect IDs (like flight booking ids) from text

I'm looking to extract ids from the body of an email. The ids are similar to flight booking IDs. For example, in an email, I would like to obtain the booking reference (something like MNFF3RGC or MNF-...
0
votes
1answer
46 views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
0
votes
1answer
37 views

Doc2Vec network architecture

I have been familiarizing myself with Word2Vec and Doc2Vec. After reading multiple papers including the the ones by T Mikolov (the creator of Doc2Vec), I am not clear on how does the neural network ...
0
votes
0answers
20 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
0
votes
1answer
41 views

what is the reason behind the bad outputs gained by RNN, LSTM when using GloVe pretrained model in text classification?

the problem is with the results gained for accuracy and f1 afer training our model via pretrained models such as GloVe. when I apply CNN as a classifier, the result are good as follows: ...
1
vote
0answers
63 views

Training of word weights in Word Embedding and Word2Vec

I want to know how are the word weights updated for the embedding layer in Keras and for Word2Vec. Like for the normal model.add(Embedding(..)) and ...
0
votes
0answers
107 views

is Glove better for word similarity Skip-gram/CBOW?

While looking at the slides for lecture 2 of CS224d: Deep Learning for Natural Language Processing: Link to slides It is said in slide number 31, that count based methods (ex: LSA) for creating word ...
1
vote
1answer
76 views

how to input the data set in to a word2vec by keras?

I am new in using word2vec model, as a result, I do not know how I can prepare my dataset as an input for word2vec? I have searched a lot but the datasets in tutorials were in CSV format or just one ...
1
vote
1answer
52 views

Would keeping all punctuation make any sense in word2vec?

I am trying to learn how word2vec works to get to more complicated stuff like LSTMs. Because I will use the same training data (so with the same vocabulary) and I want to predict punctuation too, I ...
0
votes
1answer
102 views

how to use (read) google pre-trained word2vec model file?

I am trying to apply open() function in keras to use Google news-vectors-negative300.bin which is a pre-trained file via word2vec such as GloVe, but after downloading GloVe it contains 4 files with ...
0
votes
0answers
54 views

Case-Sensitive Word Embeddings for French

Are there any pre-trained case-sensitive word embeddings for French? The only word embeddings for French I have found is FastText and it is not case sensitive. I am currently working on problems ...
0
votes
0answers
18 views

Improving clustering by incrementing density and number of elements in clusters?

Question I would like to know if, in general, clustering tecniques work better if the clusters have more elements related to them. I know that adding noise is a risk and would result in creating ...
1
vote
0answers
13 views

Are there any paper for a closed domain conversational agent

i was trying to find a closed domain conversational agent/chatbot paper in Question and answering so not long conversation, and i don't think i see any. All the paper i can find are related to an ...
1
vote
2answers
38 views

Clustering with arrays / vectors as features?

I am trying to cluster a large set of documents of which I have a DOC2VEC representation. But I want to cluster them with more features, thus resulting in having both a vector (...
1
vote
0answers
21 views

How is determined the context's dimension in Doc2Vec?

I would like to know how is determined the dimension of the context in Gensim Doc2Vec.
1
vote
0answers
74 views

Automatic Semantic Clustering and Tagging of sentences using NLP

NLP Analysis for keyword clustering I have a set of keywords for search engines and I would like to create a python script to classify and tag them under unknown categories. To make it clear I ...
2
votes
0answers
144 views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
0
votes
1answer
129 views

What is the proper train data format in LSTM?

I want to train a model to detect wrong word using in sentence. I have 1 million sentences(word base or char base) with different length. Each position(word or char) has a label to indicate it is ...
4
votes
2answers
97 views

Automatic code checking

I have some experience in machine learning, mainly clustering and classifiers. However, I am somewhat of a newbie when it comes to NLP. That said I am aware of all the various issues and ...
4
votes
1answer
402 views

Accuracy and loss don't change in CNN. Is it over-fitting?

My task is to perform classify news articles as Interesting [1] or Uninteresting [0]. My training set has 4053 articles out of which 179 are Interesting. The validation set has 664 articles out of ...
3
votes
0answers
18 views

In Word2Vec, how to the vector values translate back to related words?

Apologies for the newbie question: I've just downloadd the GoogleNews word2vec bin file and used convertvec to convert it to a text file to look at the vector values. I see they are values between -1 ...
2
votes
2answers
144 views

Image Embeddings - Negative Sampling and Imbalanced Class Issues

I am using the negative sampling approach used in Word2Vec to train some image embeddings. From what I have read, for every positive example, we are creating a number of negative examples. Question: ...
1
vote
0answers
31 views

Question about CBOW prediction?

I have a question about CBOW prediction. Suppose my job is to use 3 surrounding words w(t-3), w(t-2), w(t-1)as input to predict one target word w(t). Once the model is trained and I want to predict a ...
4
votes
1answer
42 views

Skipgram - multiple formulations?

I've been reading about the Skipgram model and I have found what I interpreted as multiple definitions. 1 - Taking a look at this blog post and Andrew Ng's Deep Learning Specialization, I understood ...
3
votes
1answer
86 views

what actually word embedding dimensions values represent?

I am learning word2vec and word embedding , I have downloaded GloVe pre-trained word embedding (shape 40,000 x 50) and using this function to extract information from that: ...
0
votes
0answers
54 views

Pre-trained Word embedding model for conversational vocabulary?

I am currently using Google’s pre-trained Word2Vec model for word sentiment analysis, however, since the model is trained on news articles I found that it's not that effective on conversational texts. ...
0
votes
2answers
418 views

How to alter word2vec wikipedia model for n-grams?

I have a very little data, so my word2vec model does not perform well. My intention is to identify words similar to technical terms such as 'support vector machine', 'machine learning', 'artificial ...