Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [word-embeddings]

Word embedding is the collective name for a set of language modeling and feature learning techniques in NLP where words are mapped to vectors of real numbers in a low dimensional space, relative to the vocabulary size.

0
votes
1answer
18 views

Alternatives to doc2vec?

What are some alternatives to the doc2vec embedding model? I.e models that convert paragraphs/documents into vectors, not just models that take the mean/sum of the word embeddings of each word in the ...
0
votes
0answers
18 views

Language Models vs Word Embeddings

I wanted to clear up my understanding of both Language Models and Word Embeddings and how they are related if at all. Language Models: There is the older counting model which consists of Markov ...
1
vote
1answer
16 views

Help with reusing glove word embedding pretrained model

When using pretrained GloVe.6B for embedding generation, How can I get only the top most frequently used 100000 words rather than all the 4M words in the file?
1
vote
1answer
24 views

Predicting topics for customer reviews based on topics mapped to n-grams?

I have a large number of unlabelled customer review data(text column) and my objective is to classify each review to a particular topic. Also I have a list of unigrams,bigrams and trigrams(not a part ...
1
vote
1answer
21 views
0
votes
0answers
12 views

Long repetitive output after changing vocabulary in seq2seq model

I trained a neural question generation model, which produces sensible questions for the vocabulary that they distributed with the paper. I wanted to run the model on a different set of word embeddings ...
0
votes
1answer
31 views

doc2vec - How does the inference step work in PV-DBOW

I am quite confused about how we generate new paragraph vectors in PV-DBOW? If I want to use the embeddings to classify some text how would I generate a vector for a new paragraph? In the original ...
1
vote
3answers
33 views

Word vectors to Sentence Vectors

How can I use the vectors of words in a sentence to get the vector of that sentence . I have used strategies like - Averaging the individual word vectors or a tf-idf weighted combination of the words ....
1
vote
1answer
21 views

What are the main distribution semantics based algorithms?

I am aware that LSI, RRI and word embeddings are distributional semantics models. However, I am not certain if the below mentioned are also distributional semantic models. Non-Negative Tensor ...
0
votes
1answer
34 views

Is skip-gram model in word2vec an expanded version of N-Gram model? skip-gram vs. skip-grams?

The skip-gram model of word2vec uses a shallow neural network to learn the word embedding with (input-word, context-word) data. When I read the tutorials for the skip-gram model there was not any ...
2
votes
1answer
36 views

Ways to Encode context for text classification?

I have a binary classification task which has the following specification: Input: Chunk of text (not more than a few sentences, mostly a sentence). Additional Input: For each input sample there is ...
1
vote
1answer
26 views

Resource and useful tips on Transfer Learning in NLP

I have a few label data for training and testing a DNN. Main purpose of my work is to train a model which can do a binary classification of text. And for this purpose, I have around 3000 label data ...
1
vote
1answer
17 views

Predicting Composition of Chemical Compounds

I have a dataset which has names of compounds and their compositions. Like below Sulphuric Acid=>[H,S,O] (Hydrogen, sulphur, oxygen) Oxalic Acid=>[H,C,O] Sodium Oxalate=>[Na,C,O] Potassium Sulphate=>[...
1
vote
1answer
68 views

How to count number of word embeddings in Gensim Word2Vec model

I am trying to create a Word2Vec model of the the Pub Med Central corpus using the Gensim library and want to limit the total number of word embeddings to around 1 billion. I have searched high and ...
2
votes
0answers
12 views

Handling large word embedding matrix in Python

I have a pre trained Glove word embedding matrix (U) of dimension (400000 * 50). Now for the purpose of query expansion I need to perform the operation matmul(U*U.T). This is the term by term ...
1
vote
0answers
16 views

Learning word embeddings using RNN

The common way of learning word embeddings is based on BOW, and Skip-gram models. Is it possible to train a RNN-based architecture like GRU or LSTM with random sentences from a large corpus to learn ...
2
votes
1answer
16 views

What is max aggregation on a set of word embeddings?

In a paper I see: $\mathcal{Q}$ is a set of words. $\psi_{G^w}$ are word embeddings. so, $\{\psi_{G^w}(w_t), \forall w_t \in \mathcal{Q}\}$ gives me a set of embeddings for all words in $\mathcal{Q}...
0
votes
0answers
21 views

Unsupervised answering for a predefined set of questions

I am working on a project to read up a text segment and find answers to a specific set of questions, in order to do some information extraction. I have a set of text corpus (each of about 3000 words),...
0
votes
1answer
46 views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
1
vote
2answers
183 views

How to retrain Glove Vectors on top of my own data?

I am using GloVe and gensim for my project. I have a corpus of data (let's say mydata.txt) which has new words which are not in the existing Glove. So, how do I retrain glove so that the existing pre-...
0
votes
1answer
41 views

what is the reason behind the bad outputs gained by RNN, LSTM when using GloVe pretrained model in text classification?

the problem is with the results gained for accuracy and f1 afer training our model via pretrained models such as GloVe. when I apply CNN as a classifier, the result are good as follows: ...
1
vote
0answers
63 views

Training of word weights in Word Embedding and Word2Vec

I want to know how are the word weights updated for the embedding layer in Keras and for Word2Vec. Like for the normal model.add(Embedding(..)) and ...
0
votes
0answers
107 views

is Glove better for word similarity Skip-gram/CBOW?

While looking at the slides for lecture 2 of CS224d: Deep Learning for Natural Language Processing: Link to slides It is said in slide number 31, that count based methods (ex: LSA) for creating word ...
1
vote
1answer
76 views

how to input the data set in to a word2vec by keras?

I am new in using word2vec model, as a result, I do not know how I can prepare my dataset as an input for word2vec? I have searched a lot but the datasets in tutorials were in CSV format or just one ...
0
votes
0answers
179 views

Enable Mini-batch Processing on PyTorch Word Embeddings

I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly. ...
1
vote
1answer
19 views

Backpropgating error to emedding matrix

I understand the backpropagation algorithm of neural networks, and how the error propagates backwards in layers. That is, I understand that given a 3-layer feed forward network, the amount to change ...
0
votes
0answers
54 views

Case-Sensitive Word Embeddings for French

Are there any pre-trained case-sensitive word embeddings for French? The only word embeddings for French I have found is FastText and it is not case sensitive. I am currently working on problems ...
1
vote
0answers
97 views

transfer learning with sentiment analysis?

The question is how good and what are some things to keep in mind when sentiment analysis models are tested on different datasets than they are trained on. Say the task is to perform sentiment ...
1
vote
1answer
44 views

Optimal Dimension of Graph(Vertex) Embedding [closed]

Let's define a embedding of a graph structure G = (V,E) where $\mid V\mid=v, \mid E \mid=e$ Now define an embedding $f: V \to R^d$ where $d\in \Bbb N$, an optimal dimension of embedding which ...
2
votes
2answers
197 views

NN embedding layer

Several neural network libraries such as tensorflow and pytorch offer an Embedding layer. Having implemented word2vec in the past, I understand the reasoning behind wanting a lower dimensional ...
1
vote
0answers
13 views

Are there any paper for a closed domain conversational agent

i was trying to find a closed domain conversational agent/chatbot paper in Question and answering so not long conversation, and i don't think i see any. All the paper i can find are related to an ...
2
votes
2answers
255 views

Initial embeddings for unknown, padding?

Last time I've been passing pretrained word embeddings into LSTM to solve text classification problems. Usually, there are additional <pad>, ...
1
vote
0answers
21 views

How is determined the context's dimension in Doc2Vec?

I would like to know how is determined the dimension of the context in Gensim Doc2Vec.
0
votes
1answer
360 views

Ratio between embedded vector dimensions and vocabulary size

Using Embedding layer in Keras on a fairly small vocabulary (~300), I am looking at how to choose the output of this layer (dense vector) when given a 300 dimension ...
-2
votes
1answer
27 views

Text + scalar features in one model

I have a set of features including text field(3-10 sentences) and about 10 scalar fields. I need to predict another scalar field (between 0 and 1). I have this field in my training/validation data. ...
1
vote
1answer
1k views

K-means clustering of word embedding gives strange results

I'm trying to cluster words based on pre trained embeddings. I ran a simple experiment where I obtained around 100 words relating to "food taste", obtained word embeddings from a pre-trained set, and ...
1
vote
0answers
30 views

NER at sentence level or document level?

Should NER models (LSTM or CRF) take input training data at sentence level or paragraph level? Let's say we have this input text, and we would like to do Named Entity Extraction: GOP Sen. Rand ...
0
votes
0answers
65 views

How to obtain word vectors for a classification task (FastText)

I a trying to implement and compare , doc2vec and fasttext, with gensim. I am quite new to it, but managed to pick up some bits from various tutorials. But I am at a point where I need to get my ...
4
votes
1answer
42 views

Skipgram - multiple formulations?

I've been reading about the Skipgram model and I have found what I interpreted as multiple definitions. 1 - Taking a look at this blog post and Andrew Ng's Deep Learning Specialization, I understood ...
3
votes
1answer
86 views

what actually word embedding dimensions values represent?

I am learning word2vec and word embedding , I have downloaded GloVe pre-trained word embedding (shape 40,000 x 50) and using this function to extract information from that: ...
0
votes
0answers
48 views

What is the intuition about output_dim in the Keras Embedding layer?

The Keras Embedding layer transforms a 2D-tensor into a 3D-tensor. This layer is often used to embed a word into a vector space of dimension output_dim (see here). ...
0
votes
0answers
54 views

Pre-trained Word embedding model for conversational vocabulary?

I am currently using Google’s pre-trained Word2Vec model for word sentiment analysis, however, since the model is trained on news articles I found that it's not that effective on conversational texts. ...
1
vote
2answers
564 views

One Hot Encoding vs Word Embeding - When to choose one or another?

A colleague of mine is having an interesting situation, he has quite a large set of possibilities for a defined categorical feature (+/- 300 different values) The usual data science approach would be ...
0
votes
2answers
418 views

How to alter word2vec wikipedia model for n-grams?

I have a very little data, so my word2vec model does not perform well. My intention is to identify words similar to technical terms such as 'support vector machine', 'machine learning', 'artificial ...
1
vote
1answer
2k views

One hot encoding vs Word embedding

I am very confused between one hot encoding and word embedding in terms of structure of the network and how it reduces the dimensionality. I am currently using encog with c# which has some ...
0
votes
0answers
43 views

Is it possible to have variable window size for Continuous Bag of Words method of training word embeddings?

All the literature I've seen so far in the CBOW model uses a fixed window size, ie window size of 2. Is it possible to have a variable window size? For example, one set will have 8 words for input ...
0
votes
0answers
166 views

How do I return Doc2Vec vectors of a corpus after training it using a pre-trained model?

I am trying to implement Doc2Vec model to convert a corpus into vectors using a pre-trained model (GoogleNews-vectors-negative300.bin). I want to return the ...
6
votes
1answer
376 views

Why do we need 2 matrices for word2vec or GloVe

Word2vec and GloVe are the two most known words embedding methods. Many works pointed that these two models are actually very close to each other and that under some assumptions, they perform a matrix ...
0
votes
0answers
46 views

keras embedding layer

I understand that given a text - [he, is, going, home] -> Keras embedding layer translates it to a vector of [24,52,54,123] as an example. However, these numbers are unscaled in the sense that going ...
1
vote
1answer
326 views

Use pretrained word vectors over custom trained word2vecs

Currently i'm working on a sentiment analysis research project using LSTM networks. As the input I convert sentences into set of vectors using word2vec. And there are some well pretrained word ...