Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Can I get un-normalized vectors from the TF USE model?

I'm using this Universal Sentence Encoder (USE) model to get embeddings of a set of texts, each text corresponding to a newspaper article. In order to build a Recommender System, I generate user ...
user avatar
1 vote
0 answers
12 views

TextVectorization and Autoencoder for feature extraction of text

I'm trying to solve a problem which is as follows: I need to train the autoencoder to extract useful data from text. I will use the trained autoencoder in another model to extract features. The goal ...
user avatar
1 vote
1 answer
24 views

Is there a sensible notion of 'character embeddings'?

There are several popular word embeddings available (e.g., Fasttext and GloVe); In short, those embeddings are a tool to encode words along with a sensible notion of semantics attached to those words (...
user avatar
1 vote
0 answers
9 views

How are the embedding and context matrices created and updated in word embedding?

I am struggling to understand how word embedding works, especially how the embedding matrix $W$ and context matrix $W'$ are created/updated. I understand that in the Input we may have a one-hot ...
user avatar
0 votes
0 answers
11 views

key generation from feature vectors in high dimentions

I welcome any suggestions to solve the following hard problem: I have a dataset of float feature vectors of size 512 where each feature vector is extracted from a face image. I want to generate a key ...
user avatar
0 votes
0 answers
43 views

Transformer time series classification using time2vec positional embedding

I want to use a transformer model to do classification of fixed-length time series. I was following along this tutorial using keras which uses time2vec as a positional embedding. According to the ...
user avatar
1 vote
1 answer
39 views

Do we perform text embedding before or after train-test splitting?

Do we perform text embedding before or after train-test splitting? I know that for encoding variables, usually done after the split. However, I'm not sure if that's also the case for text processing?
user avatar
  • 359
1 vote
0 answers
9 views

Dicussion for X-vectors

I am posting this question to ask some questions regarding the X-vector embedding proposed by Synder et al. The paper is X-VECTORS: ROBUST DNN EMBEDDINGS FOR SPEAKER RECOGNITION With reference to the ...
user avatar
  • 115
0 votes
0 answers
8 views

The output of CBOW, compared to Skipgram

From my undertanding the desired outputs from skipgram is actually the word embedding for a word, as pointed in red in the picture. But how about CBOW? Is the goal of CBOW training also aim at the ...
user avatar
  • 359
1 vote
1 answer
48 views

What is the meaning of two embedding layers in a row?

I've noticed in one deep pre-trained textual neural network that there are two embedding layers in the beginning and I don't quite understand why there are two of them. As far as I understand (correct ...
user avatar
  • 157
0 votes
0 answers
14 views

Is it possible to resize or compress word embeddings?

I have a 600 dimension GloVe word embedding with me, pretrained on a set of documents which suits my use case. However, I would like to reduce the number of embeddings to 200 dimensions. Retraining ...
user avatar
1 vote
0 answers
30 views

Graph Neural Network | How node embeddings are learned from several graphs?

I am reading paper on MEGnet which is a GNN. The objective is that we have several molecules that share same elements such as molecules $C0_2$ and $COOH$ share $C$ and $O$. Now if we learn the node ...
user avatar
  • 135
1 vote
1 answer
90 views

How can I get the output of a Keras LSTM layer?

I want to get the output (that is a vector) of a LSTM layer of a network built in Python using Keras and that is trained to classify sentences (i.e. sequences). How can I do it ? My attempt has been ...
user avatar
1 vote
1 answer
26 views

Latent space vs Embedding space | Are they same?

I am going through variational autoencoders and it is mentioned that: continuity (two close points in the latent space should not give two completely different contents once decoded) and completeness ...
user avatar
  • 135
1 vote
1 answer
16 views

Categorical Variable Embedding

I have a categorical variable in my labeled dataset. I trained one-hot encoded version of it in another neural network having embedding layer with a larger labeled dataset. I have obtained the weights ...
user avatar
  • 11
0 votes
0 answers
41 views

Stacking models using keras.layers.Concatenate with different input shapes

I have concatenated two models that uses different inputs. The first model uses input of shape (1, 33). The second model uses a feature set of dimension (1, 1024). I have a mapping function that ...
user avatar
0 votes
1 answer
136 views

Sequence Embedding using embedding layer: how does the network architecture influence it? [closed]

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
user avatar
0 votes
0 answers
28 views

mismatch between shapes using BERT

I used PyTorch to get word-embeddings using BERT for my sentences which are 150074 sentences....
user avatar
0 votes
2 answers
37 views

zero padding problem [closed]

i need to implement this code by using padding ...
user avatar
0 votes
2 answers
115 views

Why does averaging word embedding vectors (exctracted from the NN embedding layer) work to represent sentences?

I'm puzzling to understand why the method of averaging word embeddings works in order to obtain sentence embedding, in particular considering the exercize of this post How to obtain vector ...
user avatar
0 votes
1 answer
24 views

Get sentence embeddings of transformer-based models

I want to get sentence embeddings of transformer-based models (Bert, Roberta, Albert, Electra...). I plan on doing mean pooling on the hidden states of the second last layer just as what bert-as-...
user avatar
  • 145
1 vote
0 answers
20 views

how to train image classification with small difference in low-level feature

I would like to build the model that capable of doing classification for example 2 classes below : I tried alexnet, resnet50, resnet18, vgg16 but seem they are failed to differentiate between this ...
user avatar
0 votes
0 answers
9 views

In the node2vec model derivation, what does it mean for node representations to be "Symmetric in Feature Space"?

The main derivation of the probabilistic model in Node2Vec goes as follows (paper available on ArXiv: https://arxiv.org/pdf/1607.00653.pdf): We formulate feature learning in networks as a maximum ...
user avatar
  • 101
1 vote
0 answers
14 views

Are GNNs/GCNs viable for graphs with no node features, with only the unique node IDs? Are they different from DeepWalk at that point?

I started to dig into GNNs for the first time and I have trouble understanding its advantages over NLP inspired embedding methods like DeepWalk and node2vec. Do GNNs only shine with node features? Or ...
user avatar
0 votes
0 answers
45 views

KNN search of high-dimensional embedding vectors

I have a large database of n-dimensional numerical vectors. Each vector is the embedding of a vertex into n-dimensional vector space. Vertices belong to a graph, and an algorithm such as node2vec or ...
user avatar
0 votes
0 answers
7 views

Can we consider Meta-features of a datasets as its embeddings?

While reading some works on meta-learning. I had this doubt. Can we consider meta-features of a dataset as it's embedding ? Given the meta-feature is a lower dimensional representation which also try ...
user avatar
0 votes
0 answers
69 views

How to build vocabulary file for NLP embeddings efficiently?

I am currently building various word embeddings for my NLP project, ranging from Word2Vec, ELMo, LINE etc. I am looking to train ELMo using AllenNLP, a Python package for NLP, using the tutorial here. ...
user avatar
0 votes
0 answers
228 views

nn.embedding alternative for float numbers

I have found this pytorch code of transformers suitable for machine translation: ...
user avatar
0 votes
0 answers
56 views

GloVe Embedding Matrix "could not broadcast input array from shape (0) into shape (300)"

I'm working on Quora Question Pairs data set. I'm trying to get embedding matrix for GloVe with the following code: ...
user avatar
  • 438
0 votes
0 answers
15 views

Converting lists of categorical data to numeric vectors with unlabeled data

I am preparing some data for an autoencoder. One of the variables, diag_codes, is a list of codes associated with each observation. They are of varying lengths but have at least one. My question is, ...
user avatar
  • 101
1 vote
1 answer
100 views

How to choose embedding size for tensorflow recommender system

I am going to build a recommender system using TensorFlow recommender and the two-tower-model. I have wondered, how to choose the size of the embedding dimension. Are there any papers on this for ...
user avatar
1 vote
0 answers
10 views

Node embeddings and densely connected graphs

I have a dataset of users and their interests (represented by categories) and I’m trying to embed the graph which results from connecting users if they have a common interest, so I’ll add an edge for ...
user avatar
0 votes
0 answers
32 views

Tensorflow Model permanently becomes corrupt when input embeddings exceed max_position_embeddings

I use Tensorflow C++ API. I have a Tensorflow model. I give some inputs to this model. There is a parameter called max_position_embeddings This parameter determines maximum acceptable input dimensions ...
user avatar
  • 101
0 votes
1 answer
215 views

Concatenate two tensors of different shape

I have two tensors: ...
user avatar
  • 155
1 vote
1 answer
347 views

Not clear about relative position bias

I've been reading the Swin Transformer paper and came across relative position bias concept. I'm not able to figure out how is it more effective than positional embeddings. I hope someone can explain ...
user avatar
  • 141
1 vote
0 answers
57 views

Embedding numerical and categorical into one vector!

I have a set of features as follows: Where gift is categorical features and value of gift is value of the gift (numerical features). Both are the promotion. The objective is that I would like to ...
user avatar
  • 21
0 votes
0 answers
9 views

Embedding data with a graphical structure

I have an $n\times p$ dataset and wish to embed each observation in a $d$ dimensional space. The trouble is, my predictors are derived from a DAG. For a simplified example, suppose the DAG is as ...
user avatar
0 votes
0 answers
25 views

Instagram Profile Similarity Features

I want to find similar IG accounts in a semantic way(not demografic like fan count, language, country,...) and thought of the following features: Post Text Similarity (Embeddings by SBERT, averaging ...
user avatar
  • 1
1 vote
1 answer
66 views

Handle with very short and very long sequences with Neural Network

I am working on multi-class problem with sequences. My dataset is composed of sequences of data with different length. E.g. 1500 labeled samples: 500 datapoint belongs to class A, 500 class B and 500 ...
user avatar
2 votes
0 answers
824 views

Combining heterogeneous numerical and text features

We want to solve a regression problem of the form "given two objects $x$ and $y$, predict their score (think about it as a similarity) $w(x,y)$". We have 2 types of features: For each ...
user avatar
  • 71
0 votes
0 answers
9 views

Latent space optimization for sketch to image translation

I've been given a task to try and use [http://www.vision.huji.ac.il/lord/][1] architecture for the task of translating sketches to images (take for example the edge2shoes dataset) Now this ...
user avatar
  • 197
1 vote
2 answers
21 views

Low-dimensional path representation learning

I have a graph (ex: map) and multiple sequences of ids representing different paths. A vertex represents a region/area An edge between 2 vertices : a crossing from a region to another A graph path (...
user avatar
0 votes
0 answers
34 views

What is the recommended embedding for a categorical variable with more than 40000 thousand categories?

I have a feature called Planning_id with more than 40000 categories. What is the recommended embedding size? I read that: embedding_dimension = # categories * 0.25 is a good rule of thumb, but I still ...
user avatar
1 vote
0 answers
33 views

In Keras, how to organize multiple input features using pre-trained embedding mapping?

Let's say the goal is to predict weather given multiple features (temp, humidity) in the past 3 days. weather (y) can be: Sunny, Cloudy, Rainy. Temp (X1) can be : Hot, Cool, Cold. Humidity (X2) can be:...
user avatar
1 vote
1 answer
25 views

Encoding a categorical variable with a few billion possible values

I am looking to train a neural network to solve a supervised classification task. But one of my input features is a categorical variable that can have more than a few billion possible values. For ...
user avatar
0 votes
0 answers
9 views

Embedding Layer for a Single Categorical Feature in NN

I have a dataset with a mixture of 4 numerical features and 20 categorical features (some binary). I have reduced most categorical features into a small number of categories or binary, so once one hot ...
user avatar
0 votes
0 answers
7 views

Train consistent embeddings using text from different domains

I would like to train text embeddings using texts from two different domains (podcast summaries and movie summaries). The embeddings should capture similarities on topics the texts talk about, but ...
user avatar
  • 101
0 votes
0 answers
11 views

Using the document embedding widget without the internet

Is there a way to run the 'Document Embedding' widget without being connected to the internet? I am forced to work on a restricted network, so I use the portable version of Orange. I'd like to: 1: ...
user avatar
3 votes
1 answer
135 views

Graph embeddings of Wikidata items

I'm trying to use PyTorch BigGraph pre-trained embeddings of Wikidata items for disambiguation. The problem is that the results I am getting by using dot (or cosine) similarity are not great. For ...
user avatar
0 votes
1 answer
35 views

Dummy vectors and performance measurement for vector search Face Recognition

I have about thousands of person face (from celebrity dataset LFW), which each person represented by 512 x 1 vector. I stored it on vector DB to build face searching system using embedded feature (...
user avatar