Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
23 views

Difference between Gensim word2vec and keras Embedding layer

I used the gensim word2vec package and Keras Embedding layer for various different projects. Then I realize they seem to do the ...
0
votes
0answers
9 views

Connection between Embedding and LSTM and Dense layer

I am building a "predict next word" model using the following model architecture. The codes fine, but I have a few questions: ...
1
vote
1answer
12 views

What is word embedding and character embedding ? Why words are represented in vector with huge size?

In NLP word embedding represent word as number but after reading many blog i found that word are represent as vectors ? so what is word embedding exactly and Why words are represented in vector and ...
0
votes
0answers
5 views

How to draw a support set when classifying using Siamese networks without performing one shot learning?

How to perform classification on a test set with Siamese networks when I cannot afford to draw the support set from the test set itself? Possible options which come to my mind are: KNN using samples ...
1
vote
0answers
20 views

Embedding representation for a document?

Is averaging sentence embeddings, the right way to get representation for documents. Say I have a list of sentence embeddings representing symptoms. A data point looks like these: x|S1,S2,S3 --> Y|D1,...
2
votes
1answer
27 views

Document embedding vs locality sensitive hashing for document clustering

I would like to compare two methods: locality sensitivity hashing and document embedding to get the similarity between two documents. Both of those methods encode information of a document in a ...
0
votes
0answers
12 views

What information is encoded in embedding vector lengths?

I have started to investigate word2vec and related embedding strategies. The word2vec training loss is a function of cosine distance and not Euclidean distance. In fact I have been reading various ...
0
votes
0answers
11 views

Selfie image embeddings

https://arxiv.org/pdf/1906.02940.pdf I have read an article and want to implement an embedding algorithm. My problem is that I do not fully understand how the classifier is built in the decoder. More ...
1
vote
1answer
48 views

Mapping one embedding to another using Deep Learning

I am trying to write a model that has the input vector of one embedding (say $E_1$) and predicts the corresponding vector in the second embedding $E_2$. Both are n-dimensional real dense vectors $\...
0
votes
1answer
24 views

Do word embeddings help with out of vocab tokens?

I am performing sentiment analysis on a custom dataset of text with Keras but am a little confused about word embeddings. I have been able to train an "Embedding" layer and have also learned to load ...
4
votes
1answer
627 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
0
votes
0answers
26 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
1
vote
0answers
43 views

Embedding Layer on unseen data

Let's say we have a categorical variables with 5 different categories (levels). I train and get a good model based on this dataset using embedding layer with, say, 3 embedding size and with some ...
1
vote
0answers
29 views

Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
2
votes
1answer
95 views

word2vec word embeddings creates very distant vectors, closest similarity is still very far

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
2
votes
0answers
127 views

Embedding variable length “multi-hot-encoded” features

How can I implement an embedding layer in Keras that takes in an input that could have a variable length? For instance, if the vocabulary was 10-long I could have inputs like: ...
1
vote
0answers
117 views

Serious doubts on Categorical embedding

I am still having many doubts about the working of categorical embedding. In particular I have 2 points not clear: 1. Are 1-Hot variables converted to a lower dim vector? 2. What target are neural ...
1
vote
0answers
19 views

How do I recommend items to out of training users based on its recent views?

I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings. Since we have a lot of users in system, I had to sample ...
0
votes
1answer
85 views

How to get vector representations(or embeddings) of time series?

Even if a time series is constructed up of numbers only, finding abstract fixed-dim vector representation would be interesting for classification/clustering purposes. As we can learn & find ...
1
vote
0answers
44 views

When should embeddings not be used for categorical data? What are their limitations?

I recently came across the concept of embeddings so the concept is still new to me, but it is my understanding that embeddings convert one-hot encoded input data into a dense vector. Vectors ...
0
votes
0answers
26 views

How to convert tf.feature_column into a tensor?

I'd like to get an average embedding to use as an input. Without feature_column, it can be done in this way (from Tensorflow: how to look up and average a different amount of embedding vectors per ...
0
votes
0answers
26 views

Why do we share parameters between two different inputs in the embeddings layer?

I noticed in some deep learning networks that have two inputs to the network, they use one embeddings layer to share the parameters between these two different inputs. As an example, in Keras: ...
-1
votes
1answer
21 views

Finding the best “depth” of ICD9 codes with pseudo-hierarchical clustering

Here is a common problem in health care modeling. Did I just invent a new algorithm or has someone already thought of this? The goal is to find the most homogeneous partition of patients by medical ...
1
vote
0answers
197 views

Glove supported languages

Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I started experimenting with words embeddings also, and I found some interesting ...
0
votes
0answers
26 views

Does sum of embeddings make sense?

Referring to the LightFM model from paper Metadata Embeddings for User and Item Cold-start Recommendations, the model tries to learn $d$-dimensional user and item feature embeddings $e_f^U$ and $e_f^...
2
votes
0answers
98 views

how to extract the Top contributing labels/words in universal-sentence-encoder-large - TransformerModel?

I'm using the universal-sentence-encoder-large (Transformer Model) encoding process for embedding and then using the embedding for Clustering - Basically for unsupervised learning. I want to get the ...
1
vote
1answer
50 views

How to compare the output of Self-organizing maps?

I am trying to simultaneously cluster and visualize text documents using Self-organizing maps. Since text documents can be represented in various ways (vector space model, GloVe etc), I am trying to ...
1
vote
1answer
20 views

Make embedding more Gaussian-like

I am trying to train a neural network to find a mapping(embedding) to a lower dimensional space. I would like for my dataset, once mapped to the lower dimensional space, to appear gaussian-like ...
1
vote
1answer
80 views

Pretrained graph embeddings

Is there any precedent for DL use of pretrained graph embeddings in a similar manner to word embeddings?
1
vote
1answer
39 views

Loss and Regularization inference

I'm building a Matrix Factorization model for MovieLens dataset with batch-wise training. Loss function for the batch: $$ L_{batch} = 1/|B|\sum_{(u,i)\in{B}}(r_{ui} - \mu - b_u - b_i - p_u^Tq_i)^2 + \...
2
votes
0answers
234 views

Regularization in Embedding models?

What is the best way to regularize latent embeddings, I have two solution in my mind but I'm not sure which one to use over other. In batch-wise training regularize over the whole embedding matrix, ...
2
votes
2answers
51 views

What NN architecture to predict fantasy character names based on description?

I would like to build a neural network to predict a fantasy character name given a description. Like 'Scar-faced long haired elf warrior' -> 'Glorfindel' I have a dataset of about 12,000 fantasy ...
0
votes
1answer
113 views

What is the position embedding code?

https://github.com/google-research/bert/blob/master/modeling.py#L491-L520 The code of BERT is one of the implementation. But it is not what I need. I search a lot but can not judge. But where is ...
0
votes
1answer
26 views

Face embedding of unseen images

I've read about FaceNet but the main problem is still unclear for me. Does embedding work on the trained images only? Or once trained on a big dataset it will readily cluster unknown faces without re-...
6
votes
1answer
3k views

Confusion about Entity Embeddings of Categorical Variables - Working Example!

Problem Statement: I have problem making the Entity Embedding of Categorical Variable works for a simple dataset. I have followed the original github, or paper, or other blogposts[1,2,or this 3], or ...
3
votes
1answer
210 views

What is the neural network architecture behind Facebook's Starspace model?

Recently, Facebook released a paper concerning a general purpose neural embedding model called StarSpace. In their paper, they explain the loss function and the training procedure of the model, but ...
2
votes
2answers
352 views

What does the embedding mean in the FaceNet?

I am reading the paper about FaceNet but I can't get what does the embedding mean in this paper? Is it a hidden layer of the deep CNN? P.S. English isn't my native language.
4
votes
1answer
410 views

Tensorflow: how to look up and average a different amount of embedding vectors per training instance, with multiple training instances per minibatch?

In a recommender system setting: let's say I want to learn to predict future item purchases based on user past purchases using an approach inspired by Youtube's recommender system: Concretely, let's ...
1
vote
2answers
52 views

Understanding Embeddings input and output sizes

I have been trying for a while to understand the dimensionality of embeddings in neural networks and I think that finally things have clicked on my brain, however I would love to check whether or not ...
1
vote
0answers
691 views

Sequence Embedding

Is there a way to get embedding for an ordered sequence of vectors? I want to get embeddings to feed them further into net i.e. train it to arbitrary loss function simultaneously for embeddings and ...
1
vote
0answers
741 views

Triplet loss - what threshold to use to detect similarity between two embeddings?

I have trained my triplet loss model using FaceNet's architecture. I used 11k hands dataset. Now I want to see how well my model performed, so I feed it 2 images of the same class and get back their ...
2
votes
2answers
352 views

Auto-Encoder to condense (pre-process) large one-hot input vectors?

In my 3D game there are 300 categories to which a creature can belong. I would like to teach my RL agent to make decisions based on its 10 closest monsters So far, my Neural Network input vector is ...
1
vote
1answer
42 views

What tasks you train with one set of features and predict with another?

The most common scenario in supervised learning is to have data points with a set of features and train a model to make classification predictions afterward. Usually, for predictions to make sense ...
2
votes
1answer
170 views

Keras: Softmax output into embedding layer

I'm trying to build an encoder-decoder network in Keras to generate a sentence of a particular style. As my problem is unsupervised i.e. I don't have the ground truths for the generated sentences, I ...
0
votes
1answer
262 views

Is there a synset for phrasal verbs?

Is there a database of phrasal verbs of similar semantics? Eg. one where querying ‘get in touch’ would return ‘get in contact with’, among other phrasal verbs of similar meanings? If not, given a ...
1
vote
1answer
182 views

Backpropgating error to emedding matrix

I understand the backpropagation algorithm of neural networks, and how the error propagates backwards in layers. That is, I understand that given a 3-layer feed forward network, the amount to change ...
3
votes
2answers
6k views

One Hot Encoding vs Word Embeding - When to choose one or another?

A colleague of mine is having an interesting situation, he has quite a large set of possibilities for a defined categorical feature (+/- 300 different values) The usual data science approach would be ...
1
vote
0answers
672 views

Embedding layers for categorical features

Is there a threshold where it is computationally more efficient than one hot encoding to create separate keras embedding layers for each categorical feature > than x categories?
1
vote
0answers
21 views

SVD and intrinsic categorization

I just recently got interested in Data Science, and just encountered concepts such as embedding from high dimensions to low dimensions. Maybe on an irrelevant note, but in this video: Geometry of ...
3
votes
0answers
195 views

How much text is enough to train a good embedding model?

I need to train a word2vec embedding model on Wikipedia articles using Gensim. Eventually, I will use the entire Wikipedia for that but for the moment, I'm doing some experimentation/optimization to ...