Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

0
votes
0answers
14 views

How to utilize dictionary data set for text classification?

I have a dataset similar to newsgroup20 for classification. With the training dataset, I have a dictionary data set that explains some jargons in the training dataset. These both are different data ...
1
vote
0answers
21 views

Embedding dimension size for a custom Word2Vec?

Are there any guidelines for choosing the embedding dimension size value in a custom Word2Vec embedding? I know that the default is 100 and that seems just as good as any. But I'm wondering if there ...
1
vote
0answers
11 views

Reason: Average word vector embedding encodes word content and word order effectively

I was going through a research paper: FINE-GRAINED ANALYSIS OF SENTENCE EMBEDDINGS USING AUXILIARY PREDICTION TASKS The key take away was Comparison of Encoder decoder and average word sentence ...
1
vote
0answers
19 views

BERT vs Word2VEC: Is bert disambuguate the meaning of the word vector?

Word2vec: Word2vec provides a vector for each token/word and those vectors encode the meaning of the word. Although those vectors are not human interpretable, the meaning of the vectors are ...
3
votes
2answers
37 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
0
votes
2answers
23 views

I have a word2vec embedding - now what?

I've always relied on the Keras embedding layer for my NLP work. But for my latest project I want to use a custom embedding layer. I have gone through the steps to create a word2vec file but now what? ...
0
votes
0answers
12 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
0
votes
1answer
22 views

How can I use all possible spelling correction of documents before clustering those documents?

I have the data set with many documents of 50 to 100 words each. I need to clean those data by correcting misspelled words in those documents. I have an algorithm which predicts possible correct ...
1
vote
1answer
21 views

how to match a sentence to a cluster of keywords?

I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below. <...
1
vote
0answers
24 views

Copying embeddings for gensim word2vec

I wanted to see if I can simply set new weights for gensim's Word2Vec without training. I get the 20 News Group data set from scikit-learn (from sklearn.datasets import fetch_20newsgroups) and trained ...
0
votes
0answers
9 views

Oddly shaped t-SNE visualizations with Word2Vec — CBOW vs Skipgram

I've been using t-SNE to graph some Gensim Word2Vec models trained on a relatively small corpus (10 epochs). For some reason, when graphed using t-SNE, the CBOW model has a cubic-like shape, whereas ...
0
votes
0answers
15 views

Doc2vec model.docvecs giving varying output

I am using doc2vec to vectorize input text. I am converting my input dataset to tagged data and giving it as input. Initially I tried with a data of 27 input text: ...
1
vote
1answer
15 views

Embedding Values in word2vec

Are the embedding values for a particular word using word2vec Skipgram model the weights of the first layer or the softmax output of the function? Does the embedding value change according to the ...
1
vote
1answer
38 views

word2vec word embeddings creates very distant vectors, closest similarity is still very far

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
2
votes
0answers
24 views

meaning of fine-tuning in nlp task

There are two types of transfer learning model. One is feature extraction, where the weights of the pre-trained model are not changed while training on the actual task and other is the weights of the ...
1
vote
1answer
27 views

Is it possible to use Word2vec for text paraphrasing?

After reading several papers I am not sure if it is possible to some how generate text with the same meaning (paraphrase it) using only Word2vec. I found out other approaches that use sequences of ...
1
vote
0answers
21 views

using word embedding features with linear prediction models

I have been seeing that word embedding features (e.g. here or there) are used on classification or regression tasks where the classifier/regressor is a linear one: e.g. Linear/Logistic Regressor or ...
0
votes
0answers
20 views

how does the final dimensions of word2vec determined

I have the following data:- 1.the finest people are those who play tennis 2.The global economy is booming at the moment due to several factors 3.The need for human rights is beneficial even for the ...
1
vote
0answers
17 views

How to create pretrained word embedding text file with additional word features

I've had an idea for using word features to improve the quality of neural machine translation. Now, I would like to create word embeddings with additional word features such as pos tag, named entity, ...
0
votes
0answers
21 views

Reading a visualization of word embeddings

For my Masters Thesis, I created a Word2Vec model. I wanted to show this image to clarify the result. But how does the mapping works to display the words in this 2D space? All words are represented ...
6
votes
3answers
238 views

Word2Vec how to choose the embedding size parameter

I'm running word2vec over collection of documents. I understand that the size of the model is the number of dimensions of the vector space that the word is embedded into. And that different dimensions ...
0
votes
0answers
22 views

Which implementation of word2vec in keras is correct

Recently I was looking into some word2vec implementation using skip-gram model in keras. I come accross two different kinds of word2vec implementation, in which their main difference lies on the way ...
0
votes
1answer
92 views

How to train an existing word2vec gensim model on new words?

According to gensim docs, you can take an existing word2vec model and further train it on new words. The training is streamed, meaning sentences can be a generator, reading input data from disk ...
0
votes
0answers
13 views

Is there a rule for deciding dictionary size for sentiment analysis with massive datasets?

I will be performing sentiment analysis on fiction. I'll be working with around 300 books of 350 pages. Before performing word2vec, can I limit the dictionary size by ignoring less frequent words? If ...
0
votes
0answers
25 views

How can I find colours in a sentence?

Given a sentence "I like blue jeans", the output should be "blue". I do not have any training data. I'll just be downloading a bunch of tweets related to a hashtag. How do I build a model for this? ...
0
votes
1answer
53 views

CNN accuracy and loss doesn't change over epochs for sentiment analysis

I am performing text classification as Good [1] or Bad [0]. The texts are preprocessed and converted to Vectors using Google Word2Vec. Further CNN architecture is used for training. I have roughly ...
0
votes
0answers
10 views

How would I use Word2Vec model to find similar terms so that I can implement semantic search in some sense

I have built the model from the corpus but the problem is the similar words coming from the model is not expected. Also, This may be a broad question but I really cannot find a source where the ...
0
votes
1answer
81 views

Generating Similar Words (or Synonyms) with Word Embeddings (Word2Vec)

We have a search engine, and when users type in Tacos, we also want to search for similar words, such as Chilis or Burritos. However, it is also possible that the user search with multiple keywords. ...
0
votes
2answers
255 views

how to create word2vec for phrases and then calculate cosine similarity

I have just started using word2vec and I have no idea how to create vectors (using word2vec) of two lists, each containing set of words and phrases and then how to calculate cosine similarity between ...
0
votes
0answers
19 views

How PV-DBOW works

The authors of the Paragraph Vector paper describe PV-DBOW with: 2.3. Paragraph Vector without word ordering: Distributed bag of words The above method considers the concatenation of the ...
1
vote
0answers
100 views

Doc2vec most similar document to a query string

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title ...
2
votes
1answer
112 views

Why does all of NLP literature use Noise contrastive estimation loss for negative sampling instead of sampled softmax loss?

A sampled softmax function is like a regular softmax but randomly selects a given number of 'negative' samples. This is difference than NCE Loss, which doesn't use a softmax at all, it uses a ...
1
vote
0answers
34 views

Can I use Gensim doc2vec model for classification new documents?

Can I use Gensim doc2vec model for classification new documents via infer_vector? All my tests gave too bad results, even for big datasets (10GB utf-8 texts)...
0
votes
0answers
41 views

Multi-class string classification

Currently working on Resume Rarser tool using doc2vec. The main assumption that I take when parsing resume is that each line of text (docx, pdf etc) contains information of one class. Although ...
1
vote
2answers
47 views

Doc2vec '-' symbol occurrence

Currently working on resume parser and struggled with embedding words with '-' symbols in them. Such as 'IT-manager'. Vector representations of these words are incorrectly classified by doc2vec. ['...
0
votes
1answer
135 views

What to use pretrained models (Glove) or train my own model?

I have been using pre-trained models such as google news or Glove 6B model but many words in my text data does not have their vectors representation in those pre trained model. So I was thinking of ...
0
votes
0answers
33 views

Learning similarity of representations

I am interested in a framework for learning the similarity of different input representations based on some common context. I have looked into word2vec, SVD and other recommender systems, which does ...
1
vote
0answers
20 views

Word2vec compact models

Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
1
vote
1answer
69 views

Document parsing modeling and approach?

I'm relatively new to data science / machine learning (yes, I know) and am experimenting with text analysis. I only want a relatively naive approach and am looking to know whether my approach is valid ...
0
votes
0answers
20 views

define sentences with messy text data

I am extracting text from various file formats: pdf, emails, word docs, text files etc. The raw data will be processed (e.g. stemmed) but it is very likely that there are no clear sentences (e.g. ...
2
votes
2answers
130 views

Using t-SNE to track progress of a word vector embedding model. Pitfalls?

I've been training a word2vec/doc2vec model on a large amount of text. I recently stumbled across the t-SNE package, and am finding it wonderful at finding hidden structure in high-dimensional data. ...
0
votes
0answers
85 views

Normalizing Jaccard similarity scores in relation to differences in document length

The Jaccard similarity of two documents A and B can be defined as the size of their intersection (how many tokens are in both docs) divided by the size of their union (total number of tokens found in ...
0
votes
1answer
198 views

Fasttext error while loading wiki pre-trained data

I am loading the model using gensim package this way: from gensim.models import FastText model = FastText.load_fasttext_format('wiki-news-300d-1M-subword.bin') ...
1
vote
0answers
25 views

Latent feature extraction using dnn and word2vec embeddings

I recently read a journal about a tag-aware recommender system. There is a part in the paper which I do not understand. They used word2vec first and using embeddings as input to a DNN to extract ...
1
vote
1answer
100 views

Word embeddings and punctuation symbols

I have a decent understanding of word embeddings (at its core, one can think of a word being converted into a vector of, say, 100 dimensions, and each dimension given a particular value... this allows ...
0
votes
1answer
27 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
1
vote
1answer
369 views

Word embeddings for Information Retrieval - Document search?

What are good ways to find for single sentence (query) the most similiar document (text). I asked myself if word vectors (weighted average of the documents) are suitable to map a single sentence to a ...
1
vote
0answers
40 views

Doc2Vec for dataset with several text fields: concatenate or separate models?

I have a dataset with several fields: description, name, header. I want to train doc2vec out of it, so that I could use vectors for classification. So I wonder, ...
1
vote
0answers
179 views

Why activation function is not needed during the runtime of an Word2Vec model

In Word2Vec trainable model, there are two different weight matrix. The matrix $W$ from input-to-hidden layer and the matrix $W'$ from hidden-to-output layer. Referring to this article, I understand ...
0
votes
1answer
32 views

Understanding word2vec vectors representation

I'm trying to obtain the word2vec representation of few words using gensim. At present, this is the model that I have: ...