Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Where can I find dataset for word analogy task?

In the paper of Word2Vec by Thomas Mikolov and others, there is a accuracy report on the full Semantic-Syntactic data set. Where I can find this dataset or a related dataset for word analogy task? ...
0
votes
0answers
10 views

Unable to learn weights of a Word2Vec model

I was going to implement a word embedding model - namely Word2Vec - by following this TensorFlow tutorial and adapting the code a little bit. Unfortunately, though, my model won't learn anything. I've ...
0
votes
0answers
11 views

How to maintain CBOW dataset dimension and fit it in Neural Network?

I am new to neural network. I'm trying to train word embeddings without using word2vec package. Using titles from reddit worldnews dataset I'm have done some CBOW representation. For window size ...
0
votes
0answers
5 views

Training data for doc2vec models, general vs specific

I have quite a general question about doc2vec models. Let's say I have a specific NLP task whose goal is to understand the similarity between two sports news articles. Now I have the option to train ...
1
vote
1answer
24 views

Metrics for unsupervised doc2vec model

I have just built a simple doc2vec model using the gensim library, pretty much followed the tutorial located here. The methods provided for checking the quality of the model are very manual and ...
1
vote
1answer
15 views

Is it meaningful to use word2vec for non-string inputs like time series analysis?

I am working on a project that detects anomalies in a time series. I wonder if I can use word2vec for anomaly detection for non-string inputs like exchange rates?
1
vote
0answers
13 views

How can one determine that Word2Vec (CBOW method) embeddings are related to each other?

I read some fascinating stuff about the potential for using the Word2Vec algorithm to speed up the pace of scientific discovery here https://www.researchgate.net/publication/...
0
votes
0answers
15 views

How to effectively tune the hyper-parameters of Gensim Doc2Vec to achieve maximum accuracy in Document Similarity problem?

I have around 20k documents with 60 - 150 words. Out of these 20K documents, there are 400 documents for which the similar document are known. These 400 documents serve as my test data. At present I ...
0
votes
0answers
6 views

Outputs of a Doc2Vec model

I have trained a Doc2Vec model and I am interested in some of the model outputs. I am specifically interested to see if it is posible to obtain the Word embeddings from the Doc2Vec model, or obtain a ...
1
vote
1answer
26 views

Word2Vec - document similarity

Lets say I have text data for different documents from 2005 - 2015. I want to compare the similarity between $t$ and $t-1$ documents. So I take the document at 2006 and compare it with the document at ...
0
votes
1answer
23 views

Does gensim use Negative sampling in Word2vec?

When I train a word2vec model in Gensim on a huge amount of words/data (let's say hundreds of thousands of word vectors), is gensim using negative sampling automatically? Alternatively, is there a ...
0
votes
0answers
24 views

Mathematic vs Neural Network Approach for creating word vectors for a corpus of text

Are there particular advantages or disadvantages for using word2vec(neural nets) rather than Pointwise Mutual Information(PMI) and Singular Value Decomposition(SVD)(mathematical approach) for the ...
0
votes
0answers
17 views

Doc2Vec for multi class classification

I am working on my first project, I am trying to predict the quality of a software specification requirements. I have 1000 requirements which have been manually labelled on a scale of 1-5 (poor-...
0
votes
0answers
33 views

Document similarity over years: TF-IDF Word2Vec, gensim

I have two documents one at time $t$ and the other at time $t+1$. I individually calculate the TF-IDF of both documents and store my results into a document term matrix. I can load both the document ...
0
votes
0answers
45 views

Advice for making word vectors from a custom corpus

I'm working to train custom word vectors on a corpus built from my company's support tickets (using gensim). I've made some strides in getting that corpus to consist primarily of natural language (...
0
votes
0answers
19 views

How is tokenization done in pretrained word2vec models supplied by Google

I came across the pre-trained word2vec supplied by google at https://code.google.com/archive/p/word2vec/ (https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit?usp=sharing) this gives a ...
2
votes
0answers
33 views

Predicting word from a set of words

My task is to predict relevant words based on a short description of an idea. for example "SQL is a domain-specific language used in programming and designed for managing data held in a relational ...
1
vote
3answers
40 views

Word embedding of a new word which was not in training

Let's say I trained a Skip-Gram model (Word2Vec) for my vocabulary of size 10,000. The representation allows me to reduce the dimension from 10,000 (one-hot-encoding) to 100 (size of hidden layer of ...
0
votes
0answers
6 views

Usage doubt of tf.nn.nce_loss!

tf.nn.nce_loss, beautifully explained here Understanding tf.nn.nce_loss() in tensorflow, but still this method always confuse me when I compare with its actual ...
0
votes
0answers
14 views

Medication relations using word2vec

I've got a task to build neural network that creates relations between standard(pre-made) dictionary of drugs and a dictionary of drugs that pharmacy would send to us. The problem lies in the fact ...
0
votes
0answers
9 views

Using word2vec with labelled data

I am working on a project which predicts the quality (rating 1-5) of system requirements. I have approximately 1000 system requirements which have been labelled 1-5 depending on the rating by an ...
0
votes
1answer
28 views

How to utilize dictionary data set for text classification?

I have a dataset similar to newsgroup20 for classification. With the training dataset, I have a dictionary data set that explains some jargons in the training dataset. These both are different data ...
1
vote
0answers
28 views

Embedding dimension size for a custom Word2Vec?

Are there any guidelines for choosing the embedding dimension size value in a custom Word2Vec embedding? I know that the default is 100 and that seems just as good as any. But I'm wondering if there ...
4
votes
1answer
80 views

Reason: Average word vector embedding encodes word content and word order effectively

I was going through a research paper: FINE-GRAINED ANALYSIS OF SENTENCE EMBEDDINGS USING AUXILIARY PREDICTION TASKS The key take away was Comparison of Encoder decoder and average word sentence ...
1
vote
0answers
116 views

BERT vs Word2VEC: Is bert disambuguate the meaning of the word vector?

Word2vec: Word2vec provides a vector for each token/word and those vectors encode the meaning of the word. Although those vectors are not human interpretable, the meaning of the vectors are ...
4
votes
2answers
196 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
0
votes
2answers
25 views

I have a word2vec embedding - now what?

I've always relied on the Keras embedding layer for my NLP work. But for my latest project I want to use a custom embedding layer. I have gone through the steps to create a word2vec file but now what? ...
0
votes
0answers
18 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
0
votes
1answer
31 views

How can I use all possible spelling correction of documents before clustering those documents?

I have the data set with many documents of 50 to 100 words each. I need to clean those data by correcting misspelled words in those documents. I have an algorithm which predicts possible correct ...
1
vote
1answer
24 views

how to match a sentence to a cluster of keywords?

I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below. <...
1
vote
0answers
25 views

Copying embeddings for gensim word2vec

I wanted to see if I can simply set new weights for gensim's Word2Vec without training. I get the 20 News Group data set from scikit-learn (from sklearn.datasets import fetch_20newsgroups) and trained ...
0
votes
0answers
18 views

Oddly shaped t-SNE visualizations with Word2Vec — CBOW vs Skipgram

I've been using t-SNE to graph some Gensim Word2Vec models trained on a relatively small corpus (10 epochs). For some reason, when graphed using t-SNE, the CBOW model has a cubic-like shape, whereas ...
0
votes
0answers
29 views

Doc2vec model.docvecs giving varying output

I am using doc2vec to vectorize input text. I am converting my input dataset to tagged data and giving it as input. Initially I tried with a data of 27 input text: ...
1
vote
1answer
20 views

Embedding Values in word2vec

Are the embedding values for a particular word using word2vec Skipgram model the weights of the first layer or the softmax output of the function? Does the embedding value change according to the ...
1
vote
1answer
47 views

word2vec word embeddings creates very distant vectors, closest similarity is still very far

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
2
votes
0answers
62 views

meaning of fine-tuning in nlp task

There are two types of transfer learning model. One is feature extraction, where the weights of the pre-trained model are not changed while training on the actual task and other is the weights of the ...
1
vote
1answer
40 views

Is it possible to use Word2vec for text paraphrasing?

After reading several papers I am not sure if it is possible to some how generate text with the same meaning (paraphrase it) using only Word2vec. I found out other approaches that use sequences of ...
1
vote
0answers
28 views

using word embedding features with linear prediction models

I have been seeing that word embedding features (e.g. here or there) are used on classification or regression tasks where the classifier/regressor is a linear one: e.g. Linear/Logistic Regressor or ...
0
votes
0answers
23 views

how does the final dimensions of word2vec determined

I have the following data:- 1.the finest people are those who play tennis 2.The global economy is booming at the moment due to several factors 3.The need for human rights is beneficial even for the ...
1
vote
0answers
21 views

How to create pretrained word embedding text file with additional word features

I've had an idea for using word features to improve the quality of neural machine translation. Now, I would like to create word embeddings with additional word features such as pos tag, named entity, ...
0
votes
0answers
26 views

Reading a visualization of word embeddings

For my Masters Thesis, I created a Word2Vec model. I wanted to show this image to clarify the result. But how does the mapping works to display the words in this 2D space? All words are represented ...
6
votes
3answers
511 views

Word2Vec how to choose the embedding size parameter

I'm running word2vec over collection of documents. I understand that the size of the model is the number of dimensions of the vector space that the word is embedded into. And that different dimensions ...
0
votes
0answers
34 views

Which implementation of word2vec in keras is correct

Recently I was looking into some word2vec implementation using skip-gram model in keras. I come accross two different kinds of word2vec implementation, in which their main difference lies on the way ...
2
votes
3answers
245 views

How to train an existing word2vec gensim model on new words?

According to gensim docs, you can take an existing word2vec model and further train it on new words. The training is streamed, meaning sentences can be a generator, reading input data from disk ...
0
votes
0answers
14 views

Is there a rule for deciding dictionary size for sentiment analysis with massive datasets?

I will be performing sentiment analysis on fiction. I'll be working with around 300 books of 350 pages. Before performing word2vec, can I limit the dictionary size by ignoring less frequent words? If ...
0
votes
0answers
27 views

How can I find colours in a sentence?

Given a sentence "I like blue jeans", the output should be "blue". I do not have any training data. I'll just be downloading a bunch of tweets related to a hashtag. How do I build a model for this? ...
0
votes
1answer
65 views

CNN accuracy and loss doesn't change over epochs for sentiment analysis

I am performing text classification as Good [1] or Bad [0]. The texts are preprocessed and converted to Vectors using Google Word2Vec. Further CNN architecture is used for training. I have roughly ...
0
votes
0answers
15 views

How would I use Word2Vec model to find similar terms so that I can implement semantic search in some sense

I have built the model from the corpus but the problem is the similar words coming from the model is not expected. Also, This may be a broad question but I really cannot find a source where the ...
0
votes
1answer
160 views

Generating Similar Words (or Synonyms) with Word Embeddings (Word2Vec)

We have a search engine, and when users type in Tacos, we also want to search for similar words, such as Chilis or Burritos. However, it is also possible that the user search with multiple keywords. ...
0
votes
2answers
678 views

how to create word2vec for phrases and then calculate cosine similarity

I have just started using word2vec and I have no idea how to create vectors (using word2vec) of two lists, each containing set of words and phrases and then how to calculate cosine similarity between ...