Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

0
votes
0answers
12 views

Which implementation of word2vec in keras is correct

Recently I was looking into some word2vec implementation using skip-gram model in keras. I come accross two different kinds of word2vec implementation, in which their main difference lies on the way ...
0
votes
1answer
16 views

How to train an existing word2vec gensim model on new words?

According to gensim docs, you can take an existing word2vec model and further train it on new words. The training is streamed, meaning sentences can be a generator, reading input data from disk ...
0
votes
0answers
9 views

Is there a rule for deciding dictionary size for sentiment analysis with massive datasets?

I will be performing sentiment analysis on fiction. I'll be working with around 300 books of 350 pages. Before performing word2vec, can I limit the dictionary size by ignoring less frequent words? If ...
0
votes
0answers
22 views

How can I find colours in a sentence?

Given a sentence "I like blue jeans", the output should be "blue". I do not have any training data. I'll just be downloading a bunch of tweets related to a hashtag. How do I build a model for this? ...
0
votes
0answers
25 views

CNN accuracy and loss doesn't change over epochs for sentiment analysis

I am performing text classification as Good [1] or Bad [0]. The texts are preprocessed and converted to Vectors using Google Word2Vec. Further CNN architecture is used for training. I have roughly ...
0
votes
0answers
9 views

How would I use Word2Vec model to find similar terms so that I can implement semantic search in some sense

I have built the model from the corpus but the problem is the similar words coming from the model is not expected. Also, This may be a broad question but I really cannot find a source where the ...
0
votes
1answer
25 views

Generating Similar Words (or Synonyms) with Word Embeddings (Word2Vec)

We have a search engine, and when users type in Tacos, we also want to search for similar words, such as Chilis or Burritos. However, it is also possible that the user search with multiple keywords. ...
0
votes
2answers
57 views

how to create word2vec for phrases and then calculate cosine similarity

I have just started using word2vec and I have no idea how to create vectors (using word2vec) of two lists, each containing set of words and phrases and then how to calculate cosine similarity between ...
0
votes
0answers
12 views

How PV-DBOW works

The authors of the Paragraph Vector paper describe PV-DBOW with: 2.3. Paragraph Vector without word ordering: Distributed bag of words The above method considers the concatenation of the ...
1
vote
0answers
34 views

Doc2vec most similar document to a query string

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title ...
2
votes
1answer
63 views

Why does all of NLP literature use Noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
1
vote
0answers
17 views

Can I use Gensim doc2vec model for classification new documents?

Can I use Gensim doc2vec model for classification new documents via infer_vector? All my tests gave too bad results, even for big datasets (10GB utf-8 texts)...
0
votes
0answers
26 views

Multi-class string classification

Currently working on Resume Rarser tool using doc2vec. The main assumption that I take when parsing resume is that each line of text (docx, pdf etc) contains information of one class. Although ...
1
vote
2answers
44 views

Doc2vec '-' symbol occurrence

Currently working on resume parser and struggled with embedding words with '-' symbols in them. Such as 'IT-manager'. Vector representations of these words are incorrectly classified by doc2vec. ['...
0
votes
1answer
40 views

What to use pretrained models (Glove) or train my own model?

I have been using pre-trained models such as google news or Glove 6B model but many words in my text data does not have their vectors representation in those pre trained model. So I was thinking of ...
0
votes
0answers
27 views

Learning similarity of representations

I am interested in a framework for learning the similarity of different input representations based on some common context. I have looked into word2vec, SVD and other recommender systems, which does ...
1
vote
0answers
15 views

Word2vec compact models

Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
1
vote
1answer
50 views

Document parsing modeling and approach?

I'm relatively new to data science / machine learning (yes, I know) and am experimenting with text analysis. I only want a relatively naive approach and am looking to know whether my approach is valid ...
0
votes
0answers
12 views

define sentences with messy text data

I am extracting text from various file formats: pdf, emails, word docs, text files etc. The raw data will be processed (e.g. stemmed) but it is very likely that there are no clear sentences (e.g. ...
2
votes
2answers
68 views

Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
0
votes
0answers
42 views

Normalizing Jaccard similarity scores in relation to differences in document length

The Jaccard similarity of two documents A and B can be defined as the size of their intersection (how many tokens are in both docs) divided by the size of their union (total number of tokens found in ...
0
votes
1answer
103 views

Fasttext error while loading wiki pre-trained data

I am loading the model using gensim package this way: from gensim.models import FastText model = FastText.load_fasttext_format('wiki-news-300d-1M-subword.bin') ...
1
vote
0answers
22 views

Latent feature extraction using dnn and word2vec embeddings

I recently read a journal about a tag-aware recommender system. There is a part in the paper which I do not understand. They used word2vec first and using embeddings as input to a DNN to extract ...
1
vote
1answer
45 views

Word embeddings and punctuation symbols

I have a decent understanding of word embeddings (at its core, one can think of a word being converted into a vector of, say, 100 dimensions, and each dimension given a particular value... this allows ...
0
votes
1answer
21 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
1
vote
1answer
166 views

Word embeddings for Information Retrieval - Document search?

What are good ways to find for single sentence (query) the most similiar document (text). I asked myself if word vectors (weighted average of the documents) are suitable to map a single sentence to a ...
1
vote
0answers
28 views

Doc2Vec for dataset with several text fields: concatenate or separate models?

I have a dataset with several fields: description, name, header. I want to train doc2vec out of it, so that I could use vectors for classification. So I wonder, ...
1
vote
0answers
109 views

Why activation function is not needed during the runtime of an Word2Vec model

In Word2Vec trainable model, there are two different weight matrix. The matrix $W$ from input-to-hidden layer and the matrix $W'$ from hidden-to-output layer. Referring to this article, I understand ...
0
votes
1answer
21 views

Understanding word2vec vectors representation

I'm trying to obtain the word2vec representation of few words using gensim. At present, this is the model that I have: ...
1
vote
1answer
18 views

Is it possible to use word2Vec to derive hyperonymy (hyponymy or ISA relation)?

It's easy to have hyperonymy in WordNet, e.g. to know that "tea" is a case of "beverage". Is it possible to use word2Vec in this way?
1
vote
2answers
81 views

Possible reasons for word2vec learning context words as most similar rather than words in similar contexts

I am observing my word2vec model learning context words as most similar rather than words in similar contexts. I don't understand why it (word2vec in general, not my model in particular) can behave ...
1
vote
2answers
25 views

Continous bag of words claimed to be unsupervised, how is it working?

I'm following these two lectures on CBOW and skip-gram word2vec models. The first is lec 12 and the next lec 13 of a deep learning series https://www.youtube.com/watch?v=syWB-YMYZvI https://www....
1
vote
1answer
45 views

sentiment analysis for multiple entry in one text

I must do sentiment analysis on a set of financial news from s&p500 for given entities (organization names), but the problem is that each news (rows in my dataset) may have more than one entity ...
0
votes
1answer
97 views

Document similarity matching between Doc2Vec documents

I am creating a Doc2Vec model out of hundreds of PDF documents. I have 17 documents that are part of this Doc2Vec that I want to use to check similarity with other documents in the Doc2Vec model. ...
1
vote
1answer
233 views

Updating Google News Word2vec Word Embedding?

Is it possible to update the Google News Word Embedding with a custom text dataset (text data pertaining to a particular domain) ? Google News Word2Vec - Word Embedding clearly helps us to come with ...
3
votes
1answer
161 views

Why ELMo's word embedding can represent the word better than glove?

I have read the code of ELMo: https://github.com/allenai/bilm-tf Based on my understanding, ELMo first init an word embedding matrix A for all the word and then ...
1
vote
0answers
38 views

Skip-thought models applied to phrases instead of sentences

My goal is to build a statistical model with domain specific phrase embeddings. To do this, I am doing research on how to build a model using skip-thought vectors, where instead of using sentence ...
2
votes
2answers
285 views

Multi Class Classification on large dataset with over 600 classes

I'm trying to train a text data for multi class classification which comprises of 1 Million rows. After cleaning the data, I'm using a sparse matrix of Word2Vec features (Feature size is 300) The ...
0
votes
1answer
321 views

Tagging documents for doc2vec

I am working on resume parsing script. I am trying to tag documents sentences with TaggedDocument function, provided by gensim. What I have managed for now is to divide every text into sentence, put ...
0
votes
1answer
57 views

What is the best way to use word2vec for bilingual text similarity?

I face a problem where I need to compute similarities over bilingual (English and French) texts. The "database" looks like this: ...
1
vote
2answers
508 views

How to implement word to word Co-occurence matrix in python

To implement co-occurence matrix in sucha a way that number of times word1 occured in context of word2 in neighbourhood of given value, lets say 5. There are 100 words and a list with 1000 sentences. ...
0
votes
1answer
88 views

Sequence models word2vec

I am working on data-set with more than 100,000 records. This is how the data looks like: ...
2
votes
1answer
79 views

Predicting topics for customer reviews based on topics mapped to n-grams?

I have a large number of unlabelled customer review data(text column) and my objective is to classify each review to a particular topic. Also I have a list of unigrams,bigrams and trigrams(not a part ...
3
votes
1answer
242 views

Creating similarity metric with Doc2Vec and additional features

I have a dataset which contains many features. Each record is company that has many features. For example... Company A: Keywords - data, big data, tableau, dashboards, etc. Industry - Information ...
1
vote
0answers
62 views

Can we use doc2vec to detect outlier documents?

I have a set of documents and I want to identify and remove the outlier documents. I am just wondering if doc2vec can be used for this task. Or are there any recently evolved, promising algorithms ...
1
vote
1answer
250 views

How to correctly pass Word2Vec vectors as input to an LSTM

I am trying to build a text classifier using lstm which, in its first layer, has weights get by a Word2Vecmodel. In order to ...
4
votes
1answer
123 views

Are word embeddings further updated during training for document classification?

I am relatively new to the area of using word embeddings in NLP tasks. From a large corpus of documents, I train word2vec word embedding vectors and afterwards I am going to use these for document ...
0
votes
1answer
571 views

doc2vec - How does the inference step work in PV-DBOW

I am quite confused about how we generate new paragraph vectors in PV-DBOW? If I want to use the embeddings to classify some text how would I generate a vector for a new paragraph? In the original ...
2
votes
3answers
345 views

Word vectors to Sentence Vectors

How can I use the vectors of words in a sentence to get the vector of that sentence . I have used strategies like - Averaging the individual word vectors or a tf-idf weighted combination of the words ....
2
votes
1answer
905 views

How to implement LSTM using Doc2Vec vectors?

I would like to build a ANN for text classification, which has an LSTM layer, and using weights obtained via a Doc2Vec model ...