Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

How to handle different categorical embedding sizes in hold out data set

I have a pytorch tabular dataset with zip code as a categorical embedding. I'm getting great results on the test set. When I go to run my hold out sample through, it errors out because I have more ...
1
vote
0answers
10 views

Discriminator of a Conditional GAN with continuous labels

OK, let's say we have well-labeled images with non-discrete labels such as brightness or size or something and we want to generate images based on it. If it were done with a discrete label it could ...
3
votes
0answers
56 views

Are there any graph embedding algorithms like this already?

I wrote an algorithm for generating node embeddings based on the graph's topology. Most of the explanation is done in the readme file and the examples. The question is: Am I reinventing the wheel? ...
0
votes
1answer
30 views

How to encode an array of categories to feed into sklearn

I'm working on a recommendation problem, broadly following the Youtube paper on theirs. Their surrogate problem is to recommend the next video a user will watch. One feature they include in their ...
0
votes
2answers
26 views

Link prediction on network embeddings

A picture is worth a thousand words, so I decided to illustrate how I imagine the procedure of link prediction on network embeddings. In the figure below the LR model stands for the "Logistic ...
0
votes
0answers
13 views

BERT for non-textual sequence data

I'm working on a deep learning solution for classifying sequence data that isn't raw text but rather entities (which have already been extracted from the text). I am currently using word2vec-style ...
1
vote
0answers
29 views

Understanding reduced dimension embedding from tabular data

Background I am working on building a collaborative filtering recommender system in Keras for a school project, following an approach from this article. The approach is to take tabular user, item ...
0
votes
0answers
95 views

How to use additional variables that are not available in test set?

I have additional variables in my dataset that are somewhat correlated to the continuous target variable, but that are completely unavailable in the test set. So, I'm wondering how the best to use ...
0
votes
0answers
9 views

FastText and CharEmbedding

i have got a question. i don't understand how to develop my DLModel. I'm working on DGA Detection with this type of dataset: ...
1
vote
1answer
89 views

Difference between Gensim word2vec and keras Embedding layer

I used the gensim word2vec package and Keras Embedding layer for various different projects. Then I realize they seem to do the ...
0
votes
1answer
25 views

Connection between Embedding and LSTM and Dense layer

I am building a "predict next word" model using the following model architecture. The codes fine, but I have a few questions: ...
1
vote
2answers
31 views

What is word embedding and character embedding ? Why words are represented in vector with huge size?

In NLP word embedding represent word as number but after reading many blog i found that word are represent as vectors ? so what is word embedding exactly and Why words are represented in vector and ...
0
votes
0answers
6 views

How to draw a support set when classifying using Siamese networks without performing one shot learning?

How to perform classification on a test set with Siamese networks when I cannot afford to draw the support set from the test set itself? Possible options which come to my mind are: KNN using samples ...
1
vote
0answers
20 views

Embedding representation for a document?

Is averaging sentence embeddings, the right way to get representation for documents. Say I have a list of sentence embeddings representing symptoms. A data point looks like these: x|S1,S2,S3 --> Y|D1,...
2
votes
1answer
35 views

Document embedding vs locality sensitive hashing for document clustering

I would like to compare two methods: locality sensitivity hashing and document embedding to get the similarity between two documents. Both of those methods encode information of a document in a ...
0
votes
0answers
14 views

What information is encoded in embedding vector lengths?

I have started to investigate word2vec and related embedding strategies. The word2vec training loss is a function of cosine distance and not Euclidean distance. In fact I have been reading various ...
0
votes
0answers
12 views

Selfie image embeddings

https://arxiv.org/pdf/1906.02940.pdf I have read an article and want to implement an embedding algorithm. My problem is that I do not fully understand how the classifier is built in the decoder. More ...
1
vote
1answer
54 views

Mapping one embedding to another using Deep Learning

I am trying to write a model that has the input vector of one embedding (say $E_1$) and predicts the corresponding vector in the second embedding $E_2$. Both are n-dimensional real dense vectors $\...
0
votes
1answer
31 views

Do word embeddings help with out of vocab tokens?

I am performing sentiment analysis on a custom dataset of text with Keras but am a little confused about word embeddings. I have been able to train an "Embedding" layer and have also learned to load ...
5
votes
1answer
947 views

What is the difference between and Embedding Layer and an Autoencoder?

I'm reading about Embedding layers, especially applied to NLP and word2vec, and they seem nothing more than an application of Autoencoders for dimensionality reduction. Are they different? If so, what ...
0
votes
0answers
30 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
1
vote
0answers
59 views

Embedding Layer on unseen data

Let's say we have a categorical variables with 5 different categories (levels). I train and get a good model based on this dataset using embedding layer with, say, 3 embedding size and with some ...
1
vote
0answers
37 views

Approximating t-SNE embeddings for out-of-sample data

I have a large amount of data which has been reduced to two dimensions using t-SNE. Additional data points keep arriving, which I would like two-dimensional embeddings for, but this cannot be achieved ...
2
votes
1answer
159 views

word2vec word embeddings creates very distant vectors, closest similarity is still very far

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
2
votes
0answers
160 views

Embedding variable length “multi-hot-encoded” features

How can I implement an embedding layer in Keras that takes in an input that could have a variable length? For instance, if the vocabulary was 10-long I could have inputs like: ...
1
vote
0answers
121 views

Serious doubts on Categorical embedding

I am still having many doubts about the working of categorical embedding. In particular I have 2 points not clear: 1. Are 1-Hot variables converted to a lower dim vector? 2. What target are neural ...
1
vote
0answers
19 views

How do I recommend items to out of training users based on its recent views?

I used Spark's ALS implementation of matrix factorization (Collaborative Filtering for Implicit Feedback) to train user and item embeddings. Since we have a lot of users in system, I had to sample ...
0
votes
1answer
102 views

How to get vector representations(or embeddings) of time series?

Even if a time series is constructed up of numbers only, finding abstract fixed-dim vector representation would be interesting for classification/clustering purposes. As we can learn & find ...
1
vote
0answers
49 views

When should embeddings not be used for categorical data? What are their limitations?

I recently came across the concept of embeddings so the concept is still new to me, but it is my understanding that embeddings convert one-hot encoded input data into a dense vector. Vectors ...
0
votes
0answers
32 views

How to convert tf.feature_column into a tensor?

I'd like to get an average embedding to use as an input. Without feature_column, it can be done in this way (from Tensorflow: how to look up and average a different amount of embedding vectors per ...
0
votes
0answers
27 views

Why do we share parameters between two different inputs in the embeddings layer?

I noticed in some deep learning networks that have two inputs to the network, they use one embeddings layer to share the parameters between these two different inputs. As an example, in Keras: ...
-1
votes
1answer
25 views

Finding the best “depth” of ICD9 codes with pseudo-hierarchical clustering

Here is a common problem in health care modeling. Did I just invent a new algorithm or has someone already thought of this? The goal is to find the most homogeneous partition of patients by medical ...
1
vote
0answers
209 views

Glove supported languages

Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I started experimenting with words embeddings also, and I found some interesting ...
0
votes
1answer
57 views

Does sum of embeddings make sense?

Referring to the LightFM model from paper Metadata Embeddings for User and Item Cold-start Recommendations, the model tries to learn $d$-dimensional user and item feature embeddings $e_f^U$ and $e_f^...
2
votes
0answers
103 views

how to extract the Top contributing labels/words in universal-sentence-encoder-large - TransformerModel?

I'm using the universal-sentence-encoder-large (Transformer Model) encoding process for embedding and then using the embedding for Clustering - Basically for unsupervised learning. I want to get the ...
1
vote
1answer
61 views

How to compare the output of Self-organizing maps?

I am trying to simultaneously cluster and visualize text documents using Self-organizing maps. Since text documents can be represented in various ways (vector space model, GloVe etc), I am trying to ...
1
vote
1answer
22 views

Make embedding more Gaussian-like

I am trying to train a neural network to find a mapping(embedding) to a lower dimensional space. I would like for my dataset, once mapped to the lower dimensional space, to appear gaussian-like ...
1
vote
1answer
92 views

Pretrained graph embeddings

Is there any precedent for DL use of pretrained graph embeddings in a similar manner to word embeddings?
1
vote
1answer
45 views

Loss and Regularization inference

I'm building a Matrix Factorization model for MovieLens dataset with batch-wise training. Loss function for the batch: $$ L_{batch} = 1/|B|\sum_{(u,i)\in{B}}(r_{ui} - \mu - b_u - b_i - p_u^Tq_i)^2 + \...
2
votes
0answers
252 views

Regularization in Embedding models?

What is the best way to regularize latent embeddings, I have two solution in my mind but I'm not sure which one to use over other. In batch-wise training regularize over the whole embedding matrix, ...
2
votes
2answers
55 views

What NN architecture to predict fantasy character names based on description?

I would like to build a neural network to predict a fantasy character name given a description. Like 'Scar-faced long haired elf warrior' -> 'Glorfindel' I have a dataset of about 12,000 fantasy ...
0
votes
1answer
132 views

What is the position embedding code?

https://github.com/google-research/bert/blob/master/modeling.py#L491-L520 The code of BERT is one of the implementation. But it is not what I need. I search a lot but can not judge. But where is ...
0
votes
1answer
28 views

Face embedding of unseen images

I've read about FaceNet but the main problem is still unclear for me. Does embedding work on the trained images only? Or once trained on a big dataset it will readily cluster unknown faces without re-...
6
votes
1answer
4k views

Confusion about Entity Embeddings of Categorical Variables - Working Example!

Problem Statement: I have problem making the Entity Embedding of Categorical Variable works for a simple dataset. I have followed the original github, or paper, or other blogposts[1,2,or this 3], or ...
3
votes
1answer
260 views

What is the neural network architecture behind Facebook's Starspace model?

Recently, Facebook released a paper concerning a general purpose neural embedding model called StarSpace. In their paper, they explain the loss function and the training procedure of the model, but ...
2
votes
2answers
364 views

What does the embedding mean in the FaceNet?

I am reading the paper about FaceNet but I can't get what does the embedding mean in this paper? Is it a hidden layer of the deep CNN? P.S. English isn't my native language.
4
votes
1answer
471 views

Tensorflow: how to look up and average a different amount of embedding vectors per training instance, with multiple training instances per minibatch?

In a recommender system setting: let's say I want to learn to predict future item purchases based on user past purchases using an approach inspired by Youtube's recommender system: Concretely, let's ...
1
vote
2answers
55 views

Understanding Embeddings input and output sizes

I have been trying for a while to understand the dimensionality of embeddings in neural networks and I think that finally things have clicked on my brain, however I would love to check whether or not ...
1
vote
0answers
725 views

Sequence Embedding

Is there a way to get embedding for an ordered sequence of vectors? I want to get embeddings to feed them further into net i.e. train it to arbitrary loss function simultaneously for embeddings and ...
1
vote
0answers
796 views

Triplet loss - what threshold to use to detect similarity between two embeddings?

I have trained my triplet loss model using FaceNet's architecture. I used 11k hands dataset. Now I want to see how well my model performed, so I feed it 2 images of the same class and get back their ...