A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [word-embeddings]

Word embedding is the collective name for a set of language modeling and feature learning techniques in NLP where words are mapped to vectors of real numbers in a low dimensional space, relative to the vocabulary size.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

How to create POS Embeddings for input?

I am trying to append POS embeddings to an input word embedding (which is from Glove). In order to do that, I need represent POS tags as embeddings. I saw the link in this channel already; however I ...
1
vote
3answers
23 views

Categorical variables with multiple entries transformed to entity embedding

I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction. Some ...
1
vote
1answer
19 views

(pre-trained) python package for semantic word similarity

I am searching for a python package that calculates the semantic similarity between words. I do not want to train a model (what most packages seem to offer) - the package should have been pre-trained ...
0
votes
0answers
15 views

How to input sentence embedding as a feature in Neural Networks?

After preprocessing, I have the pandas dataframe like in the screenshot attached. I want to train a Keras neural network model to classify the vectors into intents. The vectors here are LASER ...
1
vote
0answers
12 views

How StarSpace works

I have gone through StarSpace algorithm but unable to understand how the output dimension is compared to Y (target variable) so that it can output the dimensions and loss function is optimized.
1
vote
1answer
19 views

How to incorporate keyboard positions on character level embeddings?

I am working with NLP and have character level embeddings. I have embeddings learned from Wikipedia text. Now, I want to learn embeddings from chat data (where misspellings and abbreviations are way ...
1
vote
1answer
22 views

How does backpropagation work with averaging layers?

I'm studying Word2Vec algorithm, and so far i understood that, in the case of input context bigger than 1 (so multiple words) we have our hidden layer that performs averaging between the inputs (as ...
2
votes
1answer
48 views

Why is T test reweighting on a word X word co-occurrence matrix so effective?

I am going through Stanford NLP class: http://web.stanford.edu/class/cs224u/ A task in the homework is to implement T-test reweighting on a word X word co-occurrence matrix: https://nbviewer.jupyter....
2
votes
1answer
31 views

Can 2 different OOV words get the same vector in FastText?

Since FastText sums up the vectors(order is not considered) of an OOV word's subwords, is it possible for two different OOV words to get the same vector ? If so, then can you give an example?
1
vote
1answer
45 views

Proper masking in the transformer model

Concerning the transformer model, a mask is used to mask out attention scores (replace with 1e-9) prior to the matrix multiplication with the value tensor. Regarding the masking, I have 3 short ...
1
vote
1answer
35 views

What's wrong with RF/SVM with word embedding (GloVe)?

I searched many times in google for examples on word embedding (specifically GloVe) with Random forest and I couldn't find any single example. For GloVe, it was all either LSTM or CNN. Maybe there's ...
5
votes
1answer
84 views

Would Topic Modelling be classified as NLP or NLU?

I recently started my journey into the world of NLP, it's been one heck of a ride. I'm currently trying to understand whether topic modelling would be considered as NLP or NLU. Initially I would ...
1
vote
0answers
12 views

Keras Embedding layer weights

I am trying to analyze the weights in a trained Keras embedding layer. The goal is to understand how well (or how poorly) we're capturing relationships between some specific tokens in our texts, ...
1
vote
3answers
37 views

group the similar words

array(['Ruby on Rails', 'Ruby', 'AWS DynamoDB', 'Python', 'MySQL', 'Swift', 'Android', 'iOS', 'JavaScript', 'React Native', 'ReactJS', 'TypeScript', 'Vue.js', 'Webpack', 'Amazon Web ...
2
votes
1answer
30 views

Difficulty interpreting word embedding vector similarity (spaCy)

I calculate vector similarities like this: ...
0
votes
0answers
8 views

Output of a seq2seq network

I've created a seq2seq lstm that uses word embeddings, following the tutorial in tutorial , but I'm having some problems understanding the output, which is along these lines: ...
2
votes
1answer
42 views

where to store embeddings for similarity search?

I've asked on stackoverflow already (here), but I figured that the approach of storing embeddings in an ordinary postgres-Database might be flawed from the very beginning. I will shortly etch out the ...
2
votes
1answer
122 views

NLP Transformers: How to get a fixed sentences embedding vectors size?

I'm loading a language model from torch hub (CamemBERT a French RoBERTa-based model) and using it do embed some sentences: ...
2
votes
1answer
24 views

Ways to combine embeddings

I am building a recommender for an online shop and I have categorical inputs that belong to one of the following categories: user current session features (e.g. ...
0
votes
2answers
36 views

How to obtain the word vectors optimally

I have a list of strings as shown sent_list = ["Carrefour is in France", "Apple pie is delicious", "Amazon has just delivered", ...] My code to get word ...
0
votes
1answer
31 views

Build a model to classify given string/text input

I need to build ML/NN model to classify/predict a given string pattern. Sample training data looks as shown in the image. Input will be the string in the column "Id Number", i need to tell to which ...
0
votes
1answer
33 views

How to encode an array of categories to feed into sklearn

I'm working on a recommendation problem, broadly following the Youtube paper on theirs. Their surrogate problem is to recommend the next video a user will watch. One feature they include in their ...
2
votes
2answers
48 views

Similarity of words using BERTMODEL

I want to find the similarity of words using the BERT model within the NER task. I have my own dataset so, I don't want to use the pre-trained model. I do the following: ...
1
vote
1answer
35 views

Do repeated sentences impact Word2Vec?

I'm working with domain-oriented documents in order to obtain synonyms using Word2Vec. These documents are usually templates, so sentences are repeated a lot. 1k of the unique sentences represent 83%...
0
votes
0answers
18 views

is it possible to implement LSTM with input shape (sample,timestep,timestep,feature)?

I'm new to Keras. I am trying to implement this model https://www.aclweb.org/anthology/D15-1167 for document classification, and I want to use LSTM for getting sentence representation. I have trained ...
0
votes
0answers
9 views

FastText and CharEmbedding

i have got a question. i don't understand how to develop my DLModel. I'm working on DGA Detection with this type of dataset: ...
1
vote
0answers
19 views

How to access an embedding table that is too large to fully load into memory?

I'm currently trying to find a way of loading/deserializing a .json file containing Flair word embeddings that is too large to fit in my RAM at once (>60GB .json with 32GB of RAM). My current code for ...
0
votes
1answer
35 views

what is sentence embeding and how to do sentence embedding for a sentence and how to use word embedding to create a sentence embedding?

what is sentence embeding ? How to do sentence embedding for sentence like example ""How old are you" ? how to use word embedding to create a sentence embedding ?
2
votes
1answer
84 views

How to build a symmetric similarity model on top of embeddings?

I have two equal length vectors that come out of two identical embedding layers. I want to calculate their similarity, and I don't trust the embedding layer enough to just use dot product (e.g. it's ...
0
votes
1answer
67 views

What does the dimension represent in the GloVe pre-trained word vectors?

I'm using GloVe pre-trained word vectors (glove.6b.50d.txt, glove.6b.300d.txt) to word embedding. I have a conceptual question: What is the difference between these files? On the other hand, what ...
1
vote
1answer
204 views

Difference between Gensim word2vec and keras Embedding layer

I used the gensim word2vec package and Keras Embedding layer for various different projects. Then I realize they seem to do the ...
0
votes
1answer
14 views

What is the defining Set in NLP

I am reading the paper Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings here is the pdf. On page 6, we read: ...
1
vote
2answers
53 views

What is word embedding and character embedding ? Why words are represented in vector with huge size?

In NLP word embedding represent word as number but after reading many blog i found that word are represent as vectors ? so what is word embedding exactly and Why words are represented in vector and ...
1
vote
1answer
46 views

How to find similar phrases

I have the following problem: I have created a customized Dictionary for getting used in some NLP tasks. I want to enhance my dictionary by finding phrases similar to the phrases I have in my ...
1
vote
0answers
17 views

Word embedding microservices in the cloud

I would like to build nlp classifiers for various tasks such as sentiment analysis, topic modeling, name entity recognition etc. . I realized that most of them only involve simple logistic regression ...
0
votes
0answers
24 views

Intuition for inference of doc2vec models, on document parts

I am trying to understand how doc2vec models perform during inference on documents when we split them in various ways. Example document: ...
0
votes
0answers
146 views

Cat2Vec embedding a categorical value columns with respect to multiple y's

I try to do some embeddings on categorical columns using Keras. Here is the code: ...
1
vote
1answer
47 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
1answer
30 views

Workaround for word embeddings that do not “see” antonyms

Most word embeddings do not "see" antonyms. For instance, among many words they will place vectors for "dependent" and "independent" (as an example) quite close, - actually as close as with synonyms ...
5
votes
2answers
593 views

Text similarity with sentence embeddings

I'm trying to calculate similarity between texts with various lengths. My current approach is following: Using Universal Sentence Encoder, I convert text to a set of vectors. I average these vectors ...
0
votes
0answers
8 views

Doubt on formulating cost function for GloVe

I'm reading the notes here and have a doubt on page 2 ("Least squares objective" section). The probability of a word $j$ occurring in the context of word $i$ is $$Q_{ij}=\frac{\exp(u_j^Tv_i)}{\sum_{w=...
2
votes
1answer
59 views

How to justify the usage of 200 dimensions in word vectors instead of the 300 dimensions?

When employing machine learning methods in NLP, most of studies use 200 or 300 dimensional vectors. 300 dimensional embeddings carry more information and this, therefore, is considered to produce ...
2
votes
0answers
24 views

NLP based Data Preprocessing Method to Improve Disease Name Prediction Using CRF and Word Embedding

I built a model( using CRF along bi lstm) to Predict New Disease Name/Entities from medical text data but the problem is Disease name appears only 5,6 times in 1 text file but on average text file ...
2
votes
1answer
75 views

Spacy word embeddings for sentence

Spacy offers pre-trained vectors for words. However I have notices that you can get vectors for sentences too: spacy_nlp('hello I').has_vector == True However I ...
0
votes
0answers
11 views

How to use SenseVector Embeddings for deep learning model?

I was facing the issue of false positives due to Word Sense Disambiguation (WSD) for text classification. For eg: 'bank' could be associated to either 'river' bank or 'commercial' bank. Using ...
0
votes
0answers
23 views

Load pretrained embedding model on TF-Hub to calculate Word Mover's Distance (WMD) on Gensim or spaCy

I'd like to calculate Word Mover's Distance with Universal Sentence Encoder on TensorFlow Hub embedding. I have tried the example on spaCy for WMD-relax, which loads 'en' model from spaCy, but I ...
1
vote
0answers
12 views

does the model update for every word encountered in word2vec?

In skipgram negative sampling according to the author's implementation, does the model update with every word? https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-...
0
votes
1answer
11 views

Models after word2vec outputs

I am originally using a bag of word (2-gram) model to approach a classification problem. The one hot encoding of the 2-gram output was sent to a logistic regression or neural network to build a ...
0
votes
0answers
25 views

Why do the training curve fall sharp suddenly?

I am training a CNN classifier on a binary balanced dataset. The dataset has 4500 numbers of tweet data along with the class of the tweet. During training, I am applying, GLOVE embedding of 300 ...
1
vote
1answer
22 views

can we make a word2vec NN of more than 3 layers using tensorflow?

To the best of my understanding , word2vec crated using gensim is of 3 layers only. I was wondering can we customize word2vec NN and create word2vec NN of more than 3 layers to experiment with it ...