Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [word-embeddings]

Word embedding is the collective name for a set of language modeling and feature learning techniques in NLP where words are mapped to vectors of real numbers in a low dimensional space, relative to the vocabulary size.

Filter by
Sorted by
Tagged with
1
vote
0answers
8 views

How to access an embedding table that is too large to fully load into memory?

I'm currently trying to find a way of loading/deserializing a .json file containing Flair word embeddings that is too large to fit in my RAM at once (>60GB .json with 32GB of RAM). My current code for ...
0
votes
1answer
10 views

what is sentence embeding and how to do sentence embedding for a sentence and how to use word embedding to create a sentence embedding?

what is sentence embeding ? How to do sentence embedding for sentence like example ""How old are you" ? how to use word embedding to create a sentence embedding ?
2
votes
1answer
78 views

How to build a symmetric similarity model on top of embeddings?

I have two equal length vectors that come out of two identical embedding layers. I want to calculate their similarity, and I don't trust the embedding layer enough to just use dot product (e.g. it's ...
0
votes
1answer
30 views

What does the dimension represent in the GloVe pre-trained word vectors?

I'm using GloVe pre-trained word vectors (glove.6b.50d.txt, glove.6b.300d.txt) to word embedding. I have a conceptual question: What is the difference between these files? On the other hand, what ...
1
vote
1answer
23 views

Difference between Gensim word2vec and keras Embedding layer

I used the gensim word2vec package and Keras Embedding layer for various different projects. Then I realize they seem to do the ...
0
votes
1answer
14 views

What is the defining Set in NLP

I am reading the paper Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings here is the pdf. On page 6, we read: ...
1
vote
1answer
12 views

What is word embedding and character embedding ? Why words are represented in vector with huge size?

In NLP word embedding represent word as number but after reading many blog i found that word are represent as vectors ? so what is word embedding exactly and Why words are represented in vector and ...
1
vote
1answer
21 views

How to find similar phrases

I have the following problem: I have created a customized Dictionary for getting used in some NLP tasks. I want to enhance my dictionary by finding phrases similar to the phrases I have in my ...
1
vote
0answers
14 views

Word embedding microservices in the cloud

I would like to build nlp classifiers for various tasks such as sentiment analysis, topic modeling, name entity recognition etc. . I realized that most of them only involve simple logistic regression ...
0
votes
0answers
17 views

Intuition for inference of doc2vec models, on document parts

I am trying to understand how doc2vec models perform during inference on documents when we split them in various ways. Example document: ...
0
votes
0answers
38 views

Cat2Vec embedding a categorical value columns with respect to multiple y's

I try to do some embeddings on categorical columns using Keras. Here is the code: ...
1
vote
1answer
36 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
1answer
25 views

Workaround for word embeddings that do not “see” antonyms

Most word embeddings do not "see" antonyms. For instance, among many words they will place vectors for "dependent" and "independent" (as an example) quite close, - actually as close as with synonyms ...
5
votes
2answers
144 views

Text similarity with sentence embeddings

I'm trying to calculate similarity between texts with various lengths. My current approach is following: Using Universal Sentence Encoder, I convert text to a set of vectors. I average these vectors ...
0
votes
0answers
7 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
1answer
53 views

How to justify the usage of 200 dimensions in word vectors instead of the 300 dimensions?

When employing machine learning methods in NLP, most of studies use 200 or 300 dimensional vectors. 300 dimensional embeddings carry more information and this, therefore, is considered to produce ...
2
votes
0answers
18 views

NLP based Data Preprocessing Method to Improve Disease Name Prediction Using CRF and Word Embedding

I built a model( using CRF along bi lstm) to Predict New Disease Name/Entities from medical text data but the problem is Disease name appears only 5,6 times in 1 text file but on average text file ...
2
votes
1answer
24 views

Spacy word embeddings for sentence

Spacy offers pre-trained vectors for words. However I have notices that you can get vectors for sentences too: spacy_nlp('hello I').has_vector == True However I ...
0
votes
0answers
8 views

How to use SenseVector Embeddings for deep learning model?

I was facing the issue of false positives due to Word Sense Disambiguation (WSD) for text classification. For eg: 'bank' could be associated to either 'river' bank or 'commercial' bank. Using ...
0
votes
0answers
11 views

Load pretrained embedding model on TF-Hub to calculate Word Mover's Distance (WMD) on Gensim or spaCy

I'd like to calculate Word Mover's Distance with Universal Sentence Encoder on TensorFlow Hub embedding. I have tried the example on spaCy for WMD-relax, which loads 'en' model from spaCy, but I ...
1
vote
0answers
11 views

does the model update for every word encountered in word2vec?

In skipgram negative sampling according to the author's implementation, does the model update with every word? https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-...
0
votes
1answer
11 views

Models after word2vec outputs

I am originally using a bag of word (2-gram) model to approach a classification problem. The one hot encoding of the 2-gram output was sent to a logistic regression or neural network to build a ...
0
votes
0answers
24 views

Why do the training curve fall sharp suddenly?

I am training a CNN classifier on a binary balanced dataset. The dataset has 4500 numbers of tweet data along with the class of the tweet. During training, I am applying, GLOVE embedding of 300 ...
1
vote
1answer
12 views

can we make a word2vec NN of more than 3 layers using tensorflow?

To the best of my understanding , word2vec crated using gensim is of 3 layers only. I was wondering can we customize word2vec NN and create word2vec NN of more than 3 layers to experiment with it ...
0
votes
1answer
16 views

Why are bigger embedding vectors not necessarily better?

I'm wondering why increasing the dimension of a word dimension vector in NLP doesn't necessarily lead to a better result. For instance, on examples I run, I see sometimes that using a pre-trained 100d ...
1
vote
1answer
42 views

Representation options of strings (keywords/topics) in models

What are all the possible ways to represent keywords in a machine learning model? The two I am aware of are: one hot encoding, using a static index. vector representation, using an embedding layer. ...
1
vote
1answer
48 views

Mapping one embedding to another using Deep Learning

I am trying to write a model that has the input vector of one embedding (say $E_1$) and predicts the corresponding vector in the second embedding $E_2$. Both are n-dimensional real dense vectors $\...
0
votes
1answer
15 views

Does the mean/median of a set sentence embedded vectors represent anything?

Please bear with me as I am new to NLP. I am specifically using tensorflow's universal sentence encoder: https://tfhub.dev/google/universal-sentence-encoder-large/3 I am clustering text based on the ...
0
votes
0answers
24 views

How does embedding layer works in pytorch with neural machine translation?

as i mentioned on title, How does pytorch embedding layer works in machine translation task ? As i know that we can use CBOW or Skip-gram to create pretrained embedding vectors for our translation ...
0
votes
0answers
24 views

Mathematic vs Neural Network Approach for creating word vectors for a corpus of text

Are there particular advantages or disadvantages for using word2vec(neural nets) rather than Pointwise Mutual Information(PMI) and Singular Value Decomposition(SVD)(mathematical approach) for the ...
1
vote
1answer
60 views

NLP - Identify Tagged Words

Please pardon me as the title might not be very accurate I am trying to create a model that learns the word representation and then is able to predict word representation in another piece of text. An ...
0
votes
0answers
30 views

Doc2Vec for multi class classification

I am working on my first project, I am trying to predict the quality of a software specification requirements. I have 1000 requirements which have been manually labelled on a scale of 1-5 (poor-...
0
votes
0answers
293 views

BERT or ELMo for Document Similarity

Does anyone use BERT or ELMo language models to determine the similarity between two text documents? My question aims to collect all possible ways for combining the contextual word embeddings ...
2
votes
0answers
49 views

Predicting word from a set of words

My task is to predict relevant words based on a short description of an idea. for example "SQL is a domain-specific language used in programming and designed for managing data held in a relational ...
1
vote
3answers
53 views

Word embedding of a new word which was not in training

Let's say I trained a Skip-Gram model (Word2Vec) for my vocabulary of size 10,000. The representation allows me to reduce the dimension from 10,000 (one-hot-encoding) to 100 (size of hidden layer of ...
0
votes
0answers
14 views

threshold for word/embeddings based on frequency in DNNLinearCombinedClassifier

I'm using Tensorflow's DNNLinearCombinedClassifier for multi-class classification. Irrespective of my vocabulary size I'm ...
0
votes
0answers
9 views

Usage doubt of tf.nn.nce_loss!

tf.nn.nce_loss, beautifully explained here Understanding tf.nn.nce_loss() in tensorflow, but still this method always confuse me when I compare with its actual ...
0
votes
1answer
24 views

Do word embeddings help with out of vocab tokens?

I am performing sentiment analysis on a custom dataset of text with Keras but am a little confused about word embeddings. I have been able to train an "Embedding" layer and have also learned to load ...
0
votes
1answer
29 views

How to utilize dictionary data set for text classification?

I have a dataset similar to newsgroup20 for classification. With the training dataset, I have a dictionary data set that explains some jargons in the training dataset. These both are different data ...
0
votes
0answers
23 views

Is Elmo equivalent to Fasttext+Bi-directional GRU?

From what I have read, Elmo uses bi-directional LSTM layers to give contextual embeddings for words in a sentence. So if I use a bi-directional LSTM/GRU layer over Fasttext representations of words, ...
4
votes
1answer
87 views

Reason: Average word vector embedding encodes word content and word order effectively

I was going through a research paper: FINE-GRAINED ANALYSIS OF SENTENCE EMBEDDINGS USING AUXILIARY PREDICTION TASKS The key take away was Comparison of Encoder decoder and average word sentence ...
0
votes
0answers
18 views

NLP text one-hot encoding with mapping dataset

I am wondering if I can make NLP model to compute word similarity with using data consisting of mapping data. I think I can make a model learn with vectorized words by one-hot encoding. Is is possible?...
1
vote
0answers
484 views

BERT vs Word2VEC: Is bert disambuguate the meaning of the word vector?

Word2vec: Word2vec provides a vector for each token/word and those vectors encode the meaning of the word. Although those vectors are not human interpretable, the meaning of the vectors are ...
4
votes
1answer
627 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
1
vote
2answers
70 views

Word Embedding for Item Names(integer, one-hot encoding)

I am looking for the way to get the similarity between two item names using integer encoding or one-hot encoding. For example, "lane connector" vs. "a truck crane". I have 100,000 item names ...
1
vote
1answer
114 views

Why use separable convolutions on one channel input?

I'm currently working on the Text Classification Guide from Google. During step 4, they create a CNN with separable convolutions for use with word embeddings: <...
1
vote
1answer
25 views

Can word embedding be used for text classification on a mix of English and non-English text?

I'm doing text classification on text messages generated by consumers and just realized even though most of the replies provided by consumers are in English, some are in French. I've used Keras word ...
0
votes
1answer
31 views

NLP: robust ways to handle morphological variations in words (e.g. plurals, verb conjugations, hyphens, etc.)?

I need to process natural language sentences in which words can appear with morphological variations: car -> cars; play -> playing, played; etc. There might be hyphens also, e.g. "dog-friendly hotel", ...
1
vote
0answers
27 views

Copying embeddings for gensim word2vec

I wanted to see if I can simply set new weights for gensim's Word2Vec without training. I get the 20 News Group data set from scikit-learn (from sklearn.datasets import fetch_20newsgroups) and trained ...
1
vote
0answers
20 views

Learning Embeddings for One Word

I have a non-conventional NLP task. I am looking to develop a sequence to a vector model. Instead of employing one-hot encoding ...