Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

Filter by
Sorted by
Tagged with
1
vote
1answer
24 views

Difference between Gensim word2vec and keras Embedding layer

I used the gensim word2vec package and Keras Embedding layer for various different projects. Then I realize they seem to do the ...
1
vote
1answer
21 views

How to find similar phrases

I have the following problem: I have created a customized Dictionary for getting used in some NLP tasks. I want to enhance my dictionary by finding phrases similar to the phrases I have in my ...
0
votes
0answers
17 views

Intuition for inference of doc2vec models, on document parts

I am trying to understand how doc2vec models perform during inference on documents when we split them in various ways. Example document: ...
0
votes
0answers
31 views

Interpreting Gensim Word2Vec Training Loss

I am using Gensim to build a Word2Vec model and identify the convergence of training loss, so that I can figure out the optimal number of iterations. For understanding this since gensim's ...
1
vote
1answer
36 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
0answers
11 views

does the model update for every word encountered in word2vec?

In skipgram negative sampling according to the author's implementation, does the model update with every word? https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-...
0
votes
1answer
11 views

Models after word2vec outputs

I am originally using a bag of word (2-gram) model to approach a classification problem. The one hot encoding of the 2-gram output was sent to a logistic regression or neural network to build a ...
1
vote
1answer
12 views

can we make a word2vec NN of more than 3 layers using tensorflow?

To the best of my understanding , word2vec crated using gensim is of 3 layers only. I was wondering can we customize word2vec NN and create word2vec NN of more than 3 layers to experiment with it ...
0
votes
1answer
16 views

Why are bigger embedding vectors not necessarily better?

I'm wondering why increasing the dimension of a word dimension vector in NLP doesn't necessarily lead to a better result. For instance, on examples I run, I see sometimes that using a pre-trained 100d ...
1
vote
2answers
44 views

Where can I find dataset for word analogy task?

In the paper of Word2Vec by Thomas Mikolov and others, there is a accuracy report on the full Semantic-Syntactic data set. Where I can find this dataset or a related dataset for word analogy task? ...
2
votes
1answer
21 views

Unable to learn weights of a Word2Vec model

I was going to implement a word embedding model - namely Word2Vec - by following this TensorFlow tutorial and adapting the code a little bit. Unfortunately, though, my model won't learn anything. I've ...
0
votes
1answer
17 views

How to maintain CBOW dataset dimension and fit it in Neural Network?

I am new to neural network. I'm trying to train word embeddings without using word2vec package. Using titles from reddit worldnews dataset I'm have done some CBOW representation. For window size ...
0
votes
0answers
5 views

Training data for doc2vec models, general vs specific

I have quite a general question about doc2vec models. Let's say I have a specific NLP task whose goal is to understand the similarity between two sports news articles. Now I have the option to train ...
1
vote
1answer
41 views

Metrics for unsupervised doc2vec model

I have just built a simple doc2vec model using the gensim library, pretty much followed the tutorial located here. The methods provided for checking the quality of the model are very manual and ...
1
vote
2answers
45 views

Is it meaningful to use word2vec for non-string inputs like time series analysis?

I am working on a project that detects anomalies in a time series. I wonder if I can use word2vec for anomaly detection for non-string inputs like exchange rates?
1
vote
0answers
13 views

How can one determine that Word2Vec (CBOW method) embeddings are related to each other?

I read some fascinating stuff about the potential for using the Word2Vec algorithm to speed up the pace of scientific discovery here https://www.researchgate.net/publication/...
1
vote
0answers
158 views

How to effectively tune the hyper-parameters of Gensim Doc2Vec to achieve maximum accuracy in Document Similarity problem?

I have around 20k documents with 60 - 150 words. Out of these 20K documents, there are 400 documents for which the similar document are known. These 400 documents serve as my test data. At present I ...
0
votes
0answers
8 views

Outputs of a Doc2Vec model

I have trained a Doc2Vec model and I am interested in some of the model outputs. I am specifically interested to see if it is posible to obtain the Word embeddings from the Doc2Vec model, or obtain a ...
1
vote
1answer
41 views

Word2Vec - document similarity

Lets say I have text data for different documents from 2005 - 2015. I want to compare the similarity between $t$ and $t-1$ documents. So I take the document at 2006 and compare it with the document at ...
0
votes
1answer
47 views

Does gensim use Negative sampling in Word2vec?

When I train a word2vec model in Gensim on a huge amount of words/data (let's say hundreds of thousands of word vectors), is gensim using negative sampling automatically? Alternatively, is there a ...
0
votes
0answers
24 views

Mathematic vs Neural Network Approach for creating word vectors for a corpus of text

Are there particular advantages or disadvantages for using word2vec(neural nets) rather than Pointwise Mutual Information(PMI) and Singular Value Decomposition(SVD)(mathematical approach) for the ...
0
votes
0answers
30 views

Doc2Vec for multi class classification

I am working on my first project, I am trying to predict the quality of a software specification requirements. I have 1000 requirements which have been manually labelled on a scale of 1-5 (poor-...
0
votes
0answers
47 views

Document similarity over years: TF-IDF Word2Vec, gensim

I have two documents one at time $t$ and the other at time $t+1$. I individually calculate the TF-IDF of both documents and store my results into a document term matrix. I can load both the document ...
0
votes
0answers
51 views

Advice for making word vectors from a custom corpus

I'm working to train custom word vectors on a corpus built from my company's support tickets (using gensim). I've made some strides in getting that corpus to consist primarily of natural language (...
0
votes
0answers
25 views

How is tokenization done in pretrained word2vec models supplied by Google

I came across the pre-trained word2vec supplied by google at https://code.google.com/archive/p/word2vec/ (https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing) this gives a ...
2
votes
0answers
49 views

Predicting word from a set of words

My task is to predict relevant words based on a short description of an idea. for example "SQL is a domain-specific language used in programming and designed for managing data held in a relational ...
1
vote
3answers
54 views

Word embedding of a new word which was not in training

Let's say I trained a Skip-Gram model (Word2Vec) for my vocabulary of size 10,000. The representation allows me to reduce the dimension from 10,000 (one-hot-encoding) to 100 (size of hidden layer of ...
0
votes
0answers
9 views

Usage doubt of tf.nn.nce_loss!

tf.nn.nce_loss, beautifully explained here Understanding tf.nn.nce_loss() in tensorflow, but still this method always confuse me when I compare with its actual ...
0
votes
0answers
16 views

Medication relations using word2vec

I've got a task to build neural network that creates relations between standard(pre-made) dictionary of drugs and a dictionary of drugs that pharmacy would send to us. The problem lies in the fact ...
0
votes
0answers
9 views

Using word2vec with labelled data

I am working on a project which predicts the quality (rating 1-5) of system requirements. I have approximately 1000 system requirements which have been labelled 1-5 depending on the rating by an ...
0
votes
1answer
29 views

How to utilize dictionary data set for text classification?

I have a dataset similar to newsgroup20 for classification. With the training dataset, I have a dictionary data set that explains some jargons in the training dataset. These both are different data ...
1
vote
0answers
64 views

Embedding dimension size for a custom Word2Vec?

Are there any guidelines for choosing the embedding dimension size value in a custom Word2Vec embedding? I know that the default is 100 and that seems just as good as any. But I'm wondering if there ...
4
votes
1answer
87 views

Reason: Average word vector embedding encodes word content and word order effectively

I was going through a research paper: FINE-GRAINED ANALYSIS OF SENTENCE EMBEDDINGS USING AUXILIARY PREDICTION TASKS The key take away was Comparison of Encoder decoder and average word sentence ...
1
vote
0answers
495 views

BERT vs Word2VEC: Is bert disambuguate the meaning of the word vector?

Word2vec: Word2vec provides a vector for each token/word and those vectors encode the meaning of the word. Although those vectors are not human interpretable, the meaning of the vectors are ...
4
votes
1answer
633 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
0
votes
2answers
27 views

I have a word2vec embedding - now what?

I've always relied on the Keras embedding layer for my NLP work. But for my latest project I want to use a custom embedding layer. I have gone through the steps to create a word2vec file but now what? ...
0
votes
0answers
26 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
0
votes
1answer
41 views

How can I use all possible spelling correction of documents before clustering those documents?

I have the data set with many documents of 50 to 100 words each. I need to clean those data by correcting misspelled words in those documents. I have an algorithm which predicts possible correct ...
1
vote
1answer
27 views

how to match a sentence to a cluster of keywords?

I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below. <...
1
vote
0answers
27 views

Copying embeddings for gensim word2vec

I wanted to see if I can simply set new weights for gensim's Word2Vec without training. I get the 20 News Group data set from scikit-learn (from sklearn.datasets import fetch_20newsgroups) and trained ...
0
votes
0answers
39 views

Oddly shaped t-SNE visualizations with Word2Vec — CBOW vs Skipgram

I've been using t-SNE to graph some Gensim Word2Vec models trained on a relatively small corpus (10 epochs). For some reason, when graphed using t-SNE, the CBOW model has a cubic-like shape, whereas ...
0
votes
0answers
49 views

Doc2vec model.docvecs giving varying output

I am using doc2vec to vectorize input text. I am converting my input dataset to tagged data and giving it as input. Initially I tried with a data of 27 input text: ...
1
vote
1answer
24 views

Embedding Values in word2vec

Are the embedding values for a particular word using word2vec Skipgram model the weights of the first layer or the softmax output of the function? Does the embedding value change according to the ...
2
votes
1answer
95 views

word2vec word embeddings creates very distant vectors, closest similarity is still very far

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
2
votes
1answer
118 views

meaning of fine-tuning in nlp task

There are two types of transfer learning model. One is feature extraction, where the weights of the pre-trained model are not changed while training on the actual task and other is the weights of the ...
1
vote
1answer
96 views

Is it possible to use Word2vec for text paraphrasing?

After reading several papers I am not sure if it is possible to some how generate text with the same meaning (paraphrase it) using only Word2vec. I found out other approaches that use sequences of ...
1
vote
0answers
34 views

using word embedding features with linear prediction models

I have been seeing that word embedding features (e.g. here or there) are used on classification or regression tasks where the classifier/regressor is a linear one: e.g. Linear/Logistic Regressor or ...
0
votes
0answers
24 views

how does the final dimensions of word2vec determined

I have the following data:- 1.the finest people are those who play tennis 2.The global economy is booming at the moment due to several factors 3.The need for human rights is beneficial even for the ...
1
vote
0answers
26 views

How to create pretrained word embedding text file with additional word features

I've had an idea for using word features to improve the quality of neural machine translation. Now, I would like to create word embeddings with additional word features such as pos tag, named entity, ...
0
votes
0answers
30 views

Reading a visualization of word embeddings

For my Masters Thesis, I created a Word2Vec model. I wanted to show this image to clarify the result. But how does the mapping works to display the words in this 2D space? All words are represented ...