Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Word embedding is the collective name for a set of language modeling and feature learning techniques in NLP where words are mapped to vectors of real numbers in a low dimensional space, relative to the vocabulary size.

0
votes
0answers
3 views

Resource and useful tips on Transfer Learning in NLP

I have a few label data for training and testing a DNN. Main purpose of my work is to train a model which can do a binary classification of text. And for this purpose, I have around 3000 label data ...
1
vote
1answer
5 views

Predicting Composition of Chemical Compounds

I have a dataset which has names of compounds and their compositions. Like below Sulphuric Acid=>[H,S,O] (Hydrogen, sulphur, oxygen) Oxalic Acid=>[H,C,O] Sodium Oxalate=>[Na,C,O] Potassium Sulphate=>[...
1
vote
1answer
8 views

How to count number of word embeddings in Gensim Word2Vec model

I am trying to create a Word2Vec model of the the Pub Med Central corpus using the Gensim library and want to limit the total number of word embeddings to around 1 billion. I have searched high and ...
2
votes
0answers
9 views

Handling large word embedding matrix in Python

I have a pre trained Glove word embedding matrix (U) of dimension (400000 * 50). Now for the purpose of query expansion I need to perform the operation matmul(U*U.T). This is the term by term ...
1
vote
0answers
9 views

Learning word embeddings using RNN

The common way of learning word embeddings is based on BOW, and Skip-gram models. Is it possible to train a RNN-based architecture like GRU or LSTM with random sentences from a large corpus to learn ...
2
votes
1answer
16 views

What is max aggregation on a set of word embeddings?

In a paper I see: $\mathcal{Q}$ is a set of words. $\psi_{G^w}$ are word embeddings. so, $\{\psi_{G^w}(w_t), \forall w_t \in \mathcal{Q}\}$ gives me a set of embeddings for all words in $\mathcal{Q}...
0
votes
0answers
18 views

Unsupervised answering for a predefined set of questions

I am working on a project to read up a text segment and find answers to a specific set of questions, in order to do some information extraction. I have a set of text corpus (each of about 3000 words),...
0
votes
1answer
28 views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
1
vote
1answer
57 views

How to retrain Glove Vectors on top of my own data?

I am using GloVe and gensim for my project. I have a corpus of data (let's say mydata.txt) which has new words which are not in the existing Glove. So, how do I retrain glove so that the existing pre-...
0
votes
1answer
27 views

what is the reason behind the bad outputs gained by RNN, LSTM when using GloVe pretrained model in text classification?

the problem is with the results gained for accuracy and f1 afer training our model via pretrained models such as GloVe. when I apply CNN as a classifier, the result are good as follows: ...
1
vote
0answers
19 views

Training of word weights in Word Embedding and Word2Vec

I want to know how are the word weights updated for the embedding layer in Keras and for Word2Vec. Like for the normal model.add(Embedding(..)) and ...
0
votes
0answers
60 views

is Glove better for word similarity Skip-gram/CBOW?

While looking at the slides for lecture 2 of CS224d: Deep Learning for Natural Language Processing: Link to slides It is said in slide number 31, that count based methods (ex: LSA) for creating word ...
0
votes
1answer
39 views

how to input the data set in to a word2vec by keras?

I am new in using word2vec model, as a result, I do not know how I can prepare my dataset as an input for word2vec? I have searched a lot but the datasets in tutorials were in CSV format or just one ...
0
votes
0answers
66 views

Enable Mini-batch Processing on PyTorch Word Embeddings

I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly. ...
1
vote
1answer
17 views

Backpropgating error to emedding matrix

I understand the backpropagation algorithm of neural networks, and how the error propagates backwards in layers. That is, I understand that given a 3-layer feed forward network, the amount to change ...
0
votes
0answers
36 views

Case-Sensitive Word Embeddings for French

Are there any pre-trained case-sensitive word embeddings for French? The only word embeddings for French I have found is FastText and it is not case sensitive. I am currently working on problems ...
1
vote
0answers
52 views

transfer learning with sentiment analysis?

The question is how good and what are some things to keep in mind when sentiment analysis models are tested on different datasets than they are trained on. Say the task is to perform sentiment ...
1
vote
1answer
42 views

Optimal Dimension of Graph(Vertex) Embedding [closed]

Let's define a embedding of a graph structure G = (V,E) where $\mid V\mid=v, \mid E \mid=e$ Now define an embedding $f: V \to R^d$ where $d\in \Bbb N$, an optimal dimension of embedding which ...
1
vote
2answers
88 views

NN embedding layer

Several neural network libraries such as tensorflow and pytorch offer an Embedding layer. Having implemented word2vec in the past, I understand the reasoning behind wanting a lower dimensional ...
1
vote
0answers
12 views

Are there any paper for a closed domain conversational agent

i was trying to find a closed domain conversational agent/chatbot paper in Question and answering so not long conversation, and i don't think i see any. All the paper i can find are related to an ...
2
votes
2answers
58 views

Initial embeddings for unknown, padding?

Last time I've been passing pretrained word embeddings into LSTM to solve text classification problems. Usually, there are additional <pad>, ...
1
vote
0answers
18 views

How is determined the context's dimension in Doc2Vec?

I would like to know how is determined the dimension of the context in Gensim Doc2Vec.
0
votes
1answer
114 views

Ratio between embedded vector dimensions and vocabulary size

Using Embedding layer in Keras on a fairly small vocabulary (~300), I am looking at how to choose the output of this layer (dense vector) when given a 300 dimension ...
-2
votes
1answer
22 views

Text + scalar features in one model

I have a set of features including text field(3-10 sentences) and about 10 scalar fields. I need to predict another scalar field (between 0 and 1). I have this field in my training/validation data. ...
-1
votes
1answer
574 views

K-means clustering of word embedding gives strange results

I'm trying to cluster words based on pre trained embeddings. I ran a simple experiment where I obtained around 100 words relating to "food taste", obtained word embeddings from a pre-trained set, and ...
1
vote
0answers
24 views

NER at sentence level or document level?

Should NER models (LSTM or CRF) take input training data at sentence level or paragraph level? Let's say we have this input text, and we would like to do Named Entity Extraction: GOP Sen. Rand ...
0
votes
0answers
55 views

How to obtain word vectors for a classification task (FastText)

I a trying to implement and compare , doc2vec and fasttext, with gensim. I am quite new to it, but managed to pick up some bits from various tutorials. But I am at a point where I need to get my ...
4
votes
1answer
41 views

Skipgram - multiple formulations?

I've been reading about the Skipgram model and I have found what I interpreted as multiple definitions. 1 - Taking a look at this blog post and Andrew Ng's Deep Learning Specialization, I understood ...
3
votes
1answer
58 views

what actually word embedding dimensions values represent?

I am learning word2vec and word embedding , I have downloaded GloVe pre-trained word embedding (shape 40,000 x 50) and using this function to extract information from that: ...
0
votes
0answers
23 views

What is the intuition about output_dim in the Keras Embedding layer?

The Keras Embedding layer transforms a 2D-tensor into a 3D-tensor. This layer is often used to embed a word into a vector space of dimension output_dim (see here). ...
0
votes
0answers
39 views

Pre-trained Word embedding model for conversational vocabulary?

I am currently using Google’s pre-trained Word2Vec model for word sentiment analysis, however, since the model is trained on news articles I found that it's not that effective on conversational texts. ...
1
vote
2answers
314 views

One Hot Encoding vs Word Embeding - When to choose one or another?

A colleague of mine is having an interesting situation, he has quite a large set of possibilities for a defined categorical feature (+/- 300 different values) The usual data science approach would be ...
0
votes
2answers
227 views

How to alter word2vec wikipedia model for n-grams?

I have a very little data, so my word2vec model does not perform well. My intention is to identify words similar to technical terms such as 'support vector machine', 'machine learning', 'artificial ...
1
vote
1answer
1k views

One hot encoding vs Word embedding

I am very confused between one hot encoding and word embedding in terms of structure of the network and how it reduces the dimensionality. I am currently using encog with c# which has some ...
0
votes
0answers
29 views

Is it possible to have variable window size for Continuous Bag of Words method of training word embeddings?

All the literature I've seen so far in the CBOW model uses a fixed window size, ie window size of 2. Is it possible to have a variable window size? For example, one set will have 8 words for input ...
0
votes
0answers
94 views

How do I return Doc2Vec vectors of a corpus after training it using a pre-trained model?

I am trying to implement Doc2Vec model to convert a corpus into vectors using a pre-trained model (GoogleNews-vectors-negative300.bin). I want to return the ...
6
votes
1answer
265 views

Why do we need 2 matrices for word2vec or GloVe

Word2vec and GloVe are the two most known words embedding methods. Many works pointed that these two models are actually very close to each other and that under some assumptions, they perform a matrix ...
0
votes
0answers
37 views

keras embedding layer

I understand that given a text - [he, is, going, home] -> Keras embedding layer translates it to a vector of [24,52,54,123] as an example. However, these numbers are unscaled in the sense that going ...
1
vote
0answers
189 views

Use pretrained word vectors over custom trained word2vecs

Currently i'm working on a sentiment analysis research project using LSTM networks. As the input I convert sentences into set of vectors using word2vec. And there are some well pretrained word ...
0
votes
0answers
35 views

How to semantically compare a paragraph to a collection of documents?

We have a collection of articles from several different online venues. We'd like to do: 1) Similarity between paragraph n to paragraph ...
1
vote
0answers
116 views

How much text is enough to train a good embedding model?

I need to train a word2vec embedding model on Wikipedia articles using gensim. Eventually, I will use the entire Wikipedia for that but for the moment, I'm doing some experimentation/optimization to ...
3
votes
1answer
496 views

Training Doc2Vec and Word2Vec at the same time

As far as I can tell the typical Doc2Vec implementation (e.g. Gensim) first trains the word vectors and afterwards the document vectors were the word vectors are fixed. If my goal is that ...
0
votes
0answers
568 views

Converting sparse IndexedSlices to a dense Tensor

I got the following warning: ...
2
votes
1answer
85 views

How to add more features in addition to a 100D word vector

I am working on sentiment analysis using tweets text. I am able to build a word vector using Keras text_to_sequence() method with pretrained GloVe embeddings in ...
1
vote
1answer
51 views

GloVe vector representation homomorphism question

In the paper GloVe: Global Vectors for Word Representation, there is this part (bottom of third page) I don't understand: I understand what groups and homomorphisms are. What I don't understand is ...
3
votes
1answer
148 views

Why do good word embeddings come out of maximizing cosine similarity?

I have an understanding of the technical details of word2vec. What I don't understand is why semantically similar words should have high cosine similarity. From what I know, goodness of a ...
5
votes
1answer
209 views

How the embedding layer is trained in Keras Embedding layer

How is the embedding layer trained in Keras Embedding layer? (say using tensorflow backend, meaning is it similar to word2vec, glove or fasttext) Assume we do not use a pretrained embedding.
0
votes
1answer
736 views

How to initialize word-embeddings for Out of Vocabulary Word?

I am trying to use CoNLL-2003 NER (English) Dataset and I am trying to utilize pretrained embeddings for it. I am using SENNA pretrained embeddings. Now I have around 20k words in my vocabulary and ...
0
votes
1answer
135 views

Reducing text input size into word2vec without affecting performance too badly?

So I am implementing Word2Vec for the first time, and I have a set of training data that I would like to train a word2vec model on. Predictably, the problem is the dataset is rather large, and I have ...
3
votes
1answer
198 views

What's the proper Word2vec model to get pre-trained word embedding for a classification task?

I have to use a neural network to classify whether some reviews of hotels are deceptive or truthful. I also have to use pre-trained word embeddings to fed the neural networks. So I can use Word2vec to ...