Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
10 views

Do word embeddings help with out of vocab tokens?

I am performing sentiment analysis on a custom dataset of text with Keras but am a little confused about word embeddings. I have been able to train an "Embedding" layer and have also learned to load ...
3
votes
2answers
64 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
0
votes
0answers
13 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
1
vote
0answers
16 views

Embedding Layer on unseen data

Let's say we have a categorical variables with 5 different categories (levels). I train and get a good model based on this dataset using embedding layer with, say, 3 embedding size and with some ...
1
vote
0answers
18 views

Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
1
vote
1answer
42 views

word2vec word embeddings creates very distant vectors, closest similarity is still very far

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
2
votes
0answers
63 views

Embedding variable length “multi-hot-encoded” features

How can I implement an embedding layer in Keras that takes in an input that could have a variable length? For instance, if the vocabulary was 10-long I could have inputs like: ...
1
vote
0answers
90 views

Serious doubts on Categorical embedding

I am still having many doubts about the working of categorical embedding. In particular I have 2 points not clear: 1. Are 1-Hot variables converted to a lower dim vector? 2. What target are neural ...
1
vote
0answers
19 views

How do I recommend items to out of training users based on its recent views?

I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings. Since we have a lot of users in system, I had to sample ...
0
votes
1answer
48 views

How to get vector representations(or embeddings) of time series?

Even if a time series is constructed up of numbers only, finding abstract fixed-dim vector representation would be interesting for classification/clustering purposes. As we can learn & find ...
1
vote
0answers
35 views

When should embeddings not be used for categorical data? What are their limitations?

I recently came across the concept of embeddings so the concept is still new to me, but it is my understanding that embeddings convert one-hot encoded input data into a dense vector. Vectors ...
0
votes
0answers
16 views

How to convert tf.feature_column into a tensor?

I'd like to get an average embedding to use as an input. Without feature_column, it can be done in this way (from Tensorflow: how to look up and average a different amount of embedding vectors per ...
0
votes
0answers
24 views

Why do we share parameters between two different inputs in the embeddings layer?

I noticed in some deep learning networks that have two inputs to the network, they use one embeddings layer to share the parameters between these two different inputs. As an example, in Keras: ...
-1
votes
1answer
16 views

Finding the best “depth” of ICD9 codes with pseudo-hierarchical clustering

Here is a common problem in health care modeling. Did I just invent a new algorithm or has someone already thought of this? The goal is to find the most homogeneous partition of patients by medical ...
1
vote
0answers
120 views

Glove supported languages

Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I started experimenting with words embeddings also, and I found some interesting ...
0
votes
0answers
21 views

Does sum of embeddings make sense?

Referring to the LightFM model from paper Metadata Embeddings for User and Item Cold-start Recommendations, the model tries to learn $d$-dimensional user and item feature embeddings $e_f^U$ and $e_f^...
2
votes
0answers
70 views

how to extract the Top contributing labels/words in universal-sentence-encoder-large - TransformerModel?

I'm using the universal-sentence-encoder-large (Transformer Model) encoding process for embedding and then using the embedding for Clustering - Basically for unsupervised learning. I want to get the ...
1
vote
1answer
44 views

How to compare the output of Self-organizing maps?

I am trying to simultaneously cluster and visualize text documents using Self-organizing maps. Since text documents can be represented in various ways (vector space model, GloVe etc), I am trying to ...
1
vote
1answer
19 views

Make embedding more Gaussian-like

I am trying to train a neural network to find a mapping(embedding) to a lower dimensional space. I would like for my dataset, once mapped to the lower dimensional space, to appear gaussian-like ...
1
vote
1answer
57 views

Pretrained graph embeddings

Is there any precedent for DL use of pretrained graph embeddings in a similar manner to word embeddings?
1
vote
1answer
33 views

Loss and Regularization inference

I'm building a Matrix Factorization model for MovieLens dataset with batch-wise training. Loss function for the batch: $$ L_{batch} = 1/|B|\sum_{(u,i)\in{B}}(r_{ui} - \mu - b_u - b_i - p_u^Tq_i)^2 + \...
2
votes
0answers
167 views

Regularization in Embedding models?

What is the best way to regularize latent embeddings, I have two solution in my mind but I'm not sure which one to use over other. In batch-wise training regularize over the whole embedding matrix, ...
2
votes
2answers
42 views

What NN architecture to predict fantasy character names based on description?

I would like to build a neural network to predict a fantasy character name given a description. Like 'Scar-faced long haired elf warrior' -> 'Glorfindel' I have a dataset of about 12,000 fantasy ...
0
votes
1answer
61 views

What is the position embedding code?

https://github.com/google-research/bert/blob/master/modeling.py#L491-L520 The code of BERT is one of the implementation. But it is not what I need. I search a lot but can not judge. But where is ...
0
votes
1answer
22 views

Face embedding of unseen images

I've read about FaceNet but the main problem is still unclear for me. Does embedding work on the trained images only? Or once trained on a big dataset it will readily cluster unknown faces without re-...
3
votes
1answer
2k views

Confusion about Entity Embeddings of Categorical Variables - Working Example!

Problem Statement: I have problem making the Entity Embedding of Categorical Variable works for a simple dataset. I have followed the original github, or paper, or other blogposts[1,2,or this 3], or ...
3
votes
1answer
137 views

What is the neural network architecture behind Facebook's Starspace model?

Recently, Facebook released a paper concerning a general purpose neural embedding model called StarSpace. In their paper, they explain the loss function and the training procedure of the model, but ...
2
votes
2answers
274 views

What does the embedding mean in the FaceNet?

I am reading the paper about FaceNet but I can't get what does the embedding mean in this paper? Is it a hidden layer of the deep CNN? P.S. English isn't my native language.
2
votes
1answer
249 views

Tensorflow: how to look up and average a different amount of embedding vectors per training instance, with multiple training instances per minibatch?

In a recommender system setting: let's say I want to learn to predict future item purchases based on user past purchases using an approach inspired by Youtube's recommender system: Concretely, let's ...
1
vote
2answers
34 views

Understanding Embeddings input and output sizes

I have been trying for a while to understand the dimensionality of embeddings in neural networks and I think that finally things have clicked on my brain, however I would love to check whether or not ...
1
vote
0answers
578 views

Sequence Embedding

Is there a way to get embedding for an ordered sequence of vectors? I want to get embeddings to feed them further into net i.e. train it to arbitrary loss function simultaneously for embeddings and ...
1
vote
0answers
579 views

Triplet loss - what threshold to use to detect similarity between two embeddings?

I have trained my triplet loss model using FaceNet's architecture. I used 11k hands dataset. Now I want to see how well my model performed, so I feed it 2 images of the same class and get back their ...
2
votes
2answers
241 views

Auto-Encoder to condense (pre-process) large one-hot input vectors?

In my 3D game there are 300 categories to which a creature can belong. I would like to teach my RL agent to make decisions based on its 10 closest monsters So far, my Neural Network input vector is ...
1
vote
1answer
37 views

What tasks you train with one set of features and predict with another?

The most common scenario in supervised learning is to have data points with a set of features and train a model to make classification predictions afterward. Usually, for predictions to make sense ...
2
votes
1answer
149 views

Keras: Softmax output into embedding layer

I'm trying to build an encoder-decoder network in Keras to generate a sentence of a particular style. As my problem is unsupervised i.e. I don't have the ground truths for the generated sentences, I ...
0
votes
1answer
209 views

Is there a synset for phrasal verbs?

Is there a database of phrasal verbs of similar semantics? Eg. one where querying ‘get in touch’ would return ‘get in contact with’, among other phrasal verbs of similar meanings? If not, given a ...
1
vote
1answer
89 views

Backpropgating error to emedding matrix

I understand the backpropagation algorithm of neural networks, and how the error propagates backwards in layers. That is, I understand that given a 3-layer feed forward network, the amount to change ...
2
votes
2answers
5k views

One Hot Encoding vs Word Embeding - When to choose one or another?

A colleague of mine is having an interesting situation, he has quite a large set of possibilities for a defined categorical feature (+/- 300 different values) The usual data science approach would be ...
1
vote
0answers
575 views

Embedding layers for categorical features

Is there a threshold where it is computationally more efficient than one hot encoding to create separate keras embedding layers for each categorical feature > than x categories?
1
vote
0answers
21 views

SVD and intrinsic categorization

I just recently got interested in Data Science, and just encountered concepts such as embedding from high dimensions to low dimensions. Maybe on an irrelevant note, but in this video: Geometry of ...
3
votes
0answers
178 views

How much text is enough to train a good embedding model?

I need to train a word2vec embedding model on Wikipedia articles using gensim. Eventually, I will use the entire Wikipedia for that but for the moment, I'm doing some experimentation/optimization to ...
1
vote
1answer
5k views

Choosing an embedding feature dimension

I'm trying to tackle a classification problem with a neural net tensor using flow. I have some continuous variable features and some categorical features. The continuous features are normalized using ...
1
vote
1answer
292 views

Categorical Variable Reduction using NN

I was trying to categorical variable engineering following this paper. The code is the following: ...
5
votes
1answer
1k views

Unordered Input

I was just wondering what the best approach is for training a neural network (or any other machine learning algorithm) where the order of the inputs does not matter. For example: f(x1,x2,x3,x4) = f(...
2
votes
1answer
2k views

How to create a multi-dimensional softmax output in Tensorflow?

I came across this research paper released by YouTube, on how they use deep learning neural networks for recommendations. It's located here: https://static.googleusercontent.com/media/research.google....
1
vote
1answer
778 views

Retain similarity distances when using an autoencoder for dimensionality reduction

I am trying to reduce the dimensionality of topic vectors (300, 1) to a two dimensional space. This has been done with various methods (e.g. t-SNE and autoencoders). A published example of reducing ...
1
vote
0answers
157 views

conceptual question about triplet loss embeddings

A question on the paper Schroff, Kalenichenko and Philbin "FaceNet: A Unified Embedding for Face Recognition and Clustering", 2015. It is my understanding that the input to the CNN is a triplet (...