Questions tagged [embeddings]
The embeddings tag has no usage guidance.
183
questions
0
votes
0
answers
10
views
For a fine-tuning a transformer to type like a specific person, should I use sentence semantic embeddings or word semantic embeddings
I'm not clear on the pros and cons of each one for this particular task. Is there even a meaningful difference?
My guess is using semantic embeddings for words will be better in nearly all cases ...
0
votes
0
answers
10
views
Handling Irregular Time Series Data with Multiple Categories in Hourly Resolution Embeddings
I'm working with a time series dataset that has 100 categories for a single feature, and I've generated embeddings for these categories. However, my problem arises when I am trying to sample the data ...
0
votes
0
answers
27
views
Is openAI text generation models an extension of embedding models?
we can creating embeddings using below code
...
- 167
0
votes
0
answers
27
views
Using Embedding For Regularization
Is using embeddings for regularization a valid practice?
My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
1
vote
1
answer
59
views
Why is T5 often used in text-to-data for text prompt encoders?
In the text-to-data(music, image, audio, etc.) generative AI field, one method of encoding text prompts is to use pre-trained language models. Such an approach was used in research on Moûsai [1] and ...
- 13
0
votes
0
answers
29
views
Combining Textual, Categorical and Numerical data for Semantic Search using SentenceTransformers model
I'm building a food semantic search model and I want to use a pre-trained SentenceTransformers model with cosine similarity. I'm using Epicurious dataset for the corpus which consists of textual (&...
- 1
0
votes
1
answer
25
views
Using text embeddings directly to compute similarity vs using them as features for a model that predicts similariy
Say you have a problem where you have a query and a set of result documents and you want to rank the result documents according to the query. Say also you have embeddings for the query and for the ...
- 183
2
votes
0
answers
48
views
CLIP Visual Transformer image encoder
I was doing some experiments with the CLIP's visual transformer encoder output (clip-ViT-B-32). So basically given the same scene or image, it should output almost ...
- 217
0
votes
1
answer
57
views
Are the terms "vector embedding" and "embedding vector" interchangeable?
Googling "vector embedding" gives me ~40k results, including from Pinecone, Weaviate, and OpenAI.
Googling ...
- 103
0
votes
0
answers
22
views
Are Vector embeddings position or direction vectors?
I've seen statements about embeddings being positions on a hypersphere. I've also seen statements about using dot product to determine similarity of these positions. This implies to me that we care ...
0
votes
0
answers
621
views
How to add knowledge to the LLM using LangChain (at a high level)?
At a super high level, I would like to create a fantasy language AI tutor. For this question, however, I would like to better understand how, generally speaking, you add your own custom data/knowledge ...
- 263
2
votes
2
answers
301
views
Why cant we use normalise position encodings instead of the cos and sine encodings used in the Transformer paper?
I'm working with Transformer models for sequence-to-sequence tasks and I'm trying to fully understand the use of positional encodings in these models.
In the original "Attention is All You Need&...
0
votes
2
answers
570
views
Recommended way to embed a text thousands of tokens long?
I've split the text up sections each 512 tokens long and created embeddings for each of them.
I want to combine them into 1 embedding for the full text. How do I do that? Is this even recommended?
...
- 268
0
votes
0
answers
64
views
Why are similarities all close to 1 in my two tower recommender?
I am trying to build a simple two tower recommender system on the MovieLens 100k dataset. The user tower is just a simple embedding layer. The item tower uses an embedding layer and concats that with ...
- 1
0
votes
1
answer
28
views
How do I interpret low dimentional embeddings of high dimentional embeddings?
I am trying to understand what I am supposed to learn about a problem when using dimensionality reduction methods. In particular, I am referring to methods like t-SNE and UMAP.
For the most part I am ...
- 111
0
votes
0
answers
14
views
High-Cardinality Categorical Feature with frequency score
I'm working with a really high-cardinality feature as one of the inputs to my model and I'm using hash-encoded feature embedding rather than one-hot encoding. However, this method is ignoring the ...
- 101
1
vote
0
answers
118
views
Otimization of similarity search for multiple embeddings by creating a weighted artificial embedding
I have embeddings of text created with a BERT model. A group of these embeddings should be used to find similar embeddings corresponding to this group. I know that you can use average or max (or ...
- 113
0
votes
0
answers
30
views
Max word embedding
I'm trying to understand the concept behind Max Word / Sentence embeddings also in contrast to mean word embedding. They used it for example in this article. So far I didn't find good resources which ...
- 113
0
votes
0
answers
43
views
Distance Metric for a dataset with embeddings and numerical columns
I'm trying to build an Approximate Nearest Neighbours model that can fetch similar records in a dataset, that are contextually similar. For example, for a record of job=Software Engineer and age=25, a ...
0
votes
0
answers
17
views
Align sub-sentences with sentences using embeddings
I would like ask for ideas
So I have sub-sentences, embedded in the context of their respective full-sentences. Then, I have other full-sentences and I would like to find
a) if they have similar sub-...
- 123
0
votes
0
answers
9
views
How to generate embedding for Image Dataset with multiple features?
My dataset has Jewellery images. The jewellery is of 4 types:
Type
Color
Shape
Material
Now I am aware of converting images to embeddings. I need to group similar images like two plain gold rings ...
- 265
0
votes
0
answers
21
views
How the embedding model (x-vectors) trained?
I read this paper: X-Vectors: Robust DNN Embeddings for Speaker Recognition which describes how PyAnnote embedding block works.
I'm not sure I understand how the X-Vector model was trained and tested:
...
- 363
0
votes
1
answer
210
views
SBERT Embeddings from Conversations
I have a dataset consisting of text-based conversations between two humans. One conversation has on average 20 turns and can look as follows:
...
- 121
0
votes
0
answers
11
views
In contrastive loss, why do we sample negatives from the document, not the query?
From what I read, for image search contrastive training, we give one query in text, and 1 positive image, and (N-1) sampled negative images.
However, for video search contrastive training, we give 1 ...
- 101
0
votes
0
answers
19
views
Does word2vec skip gram involves softmax in the output layer
I was going through various pytorch and from-scratch implementations of skip-gram. I found following:
This implementaiton does not seem to use softmax
...
- 103
1
vote
1
answer
41
views
Visualizing convolutional neural networks embedding
In this article, the author creates a graph (at the end of the post) from the embeddings of different words found by transformer model. I would like to do a similar thing for a convolutional neural ...
- 113
0
votes
0
answers
36
views
Inductive embedding technique for nodes without features
I am working on a project that involves graph embeddings. I have an homogeneous directed graph where the nodes have no features. The only available information are the edges between nodes. I need to ...
- 1
0
votes
1
answer
29
views
Is there a way to embed curves into smaller vectors?
I know about embeddings for words, but I would like to know if it is possible to do something similar for curves. What I mean by curves is a curve of a function. Say I have 1000 points corresponding ...
- 1
0
votes
0
answers
24
views
Select Topic Words from Clusters
I am following this solution for clustering: https://towardsdatascience.com/clustering-contextual-embeddings-for-topic-model-1fb15c45b1bd
For step four "Select Topic Words from Clusters", I ...
- 129
1
vote
1
answer
39
views
Which pre-trained model to select to generate embeddings from shop names written in English?
Good afternoon!
I have a dataset with thousands of shop names written in English. Several shop names might belong to one business entity, for instance, shops with names "KFC 001", "WWW....
- 11
0
votes
0
answers
32
views
Embedding: Can i use it in a time series problem?
I'm trying to do feature extraction in some discretized time series with a variable length, doing that i'm creating an RNN auto encoder. My main problem is to find a way to let the model train with ...
- 21
0
votes
1
answer
21
views
Are there methods to represent entire knowledge graphs using a single embedding vector?
In a knowledge graph, embedding vectors can be learned for nodes (node embedding) and edges (edge embeddings). Is there a method to learn one single embedding vector for the entire knowledge graph?
- 11
1
vote
0
answers
68
views
Machine learning with mixed variables dataset (numerical, categorical and embeddings)
I'm working on a machine learning project where I'm trying to predict the revenue of a movie.
My dataset contains mixed data types. There are numerical features (rating, number of votes, release year,....
- 111
1
vote
0
answers
110
views
Embedding performing unexpectedly worse than multi-hot encoding
I'm pretty new to ML and have set up a Keras model with a number of categorical features of users & items to predict positive reactions.
Each feature is a list of IDs from a database - i.e. ...
- 11
0
votes
0
answers
54
views
Practical application of denoising autoencoders
I have been reading into autoencoders for the purpose of denoising data. In the examples i found (eg. [1, 2, 3], which are the first few google results) they have the following input/output:
Input ...
1
vote
1
answer
54
views
Understand the reason of embedding and the size inside it in Pytorch
I'm very new to pytorch - taking a course in udemy.
There is something I find hard to understand and would like to get explaination about, in a bit simpler words than what I can find in the ...
- 349
1
vote
1
answer
63
views
Biometrics identification with embeddings comparison and "unknown"/"other" class/label
This is a general or more conceptual questions about biometric classification models, based on deep learning neural networks.
The goal of the system is to take a set of features (e.g. voice recording, ...
- 123
1
vote
1
answer
79
views
Product embeddings
Hei, I have a list of purchase baskets from customers and would like to build embeddings for the products.
For example:
BASKET1 = ['PRODUCT234', 'PRODUCT214', 'PRODUCT768']
BASKET2 = ['PRODUCT2', '...
- 145
1
vote
1
answer
139
views
Is there a sensible notion of 'character embeddings'?
There are several popular word embeddings available (e.g., Fasttext and GloVe); In short, those embeddings are a tool to encode words along with a sensible notion of semantics attached to those words (...
- 123
1
vote
0
answers
242
views
How are the embedding and context matrices created and updated in word embedding?
I am struggling to understand how word embedding works, especially how the embedding matrix $W$ and context matrix $W'$ are created/updated. I understand that in the Input we may have a one-hot ...
2
votes
1
answer
2k
views
Transformer time series classification using time2vec positional embedding
I want to use a transformer model to do classification of fixed-length time series. I was following along this tutorial using keras which uses time2vec as a positional embedding. According to the ...
- 123
1
vote
1
answer
647
views
Do we perform text embedding before or after train-test splitting?
Do we perform text embedding before or after train-test splitting?
I know that for encoding variables, usually done after the split. However, I'm not sure if that's also the case for text processing?
- 411
2
votes
0
answers
16
views
Dicussion for X-vectors
I am posting this question to ask some questions regarding the X-vector embedding proposed by Synder et al.
The paper is X-VECTORS: ROBUST DNN EMBEDDINGS FOR SPEAKER RECOGNITION
With reference to the ...
- 125
1
vote
1
answer
473
views
What is the meaning of two embedding layers in a row?
I've noticed in one deep pre-trained textual neural network that there are two embedding layers in the beginning and I don't quite understand why there are two of them. As far as I understand (correct ...
- 157
1
vote
0
answers
76
views
Graph Neural Network | How node embeddings are learned from several graphs?
I am reading paper on MEGnet which is a GNN. The objective is that we have several molecules that share same elements such as molecules $C0_2$ and $COOH$ share $C$ and $O$. Now if we learn the node ...
- 155
1
vote
1
answer
2k
views
How can I get the output of a Keras LSTM layer?
I want to get the output (that is a vector) of a LSTM layer of a network built in Python using Keras and that is trained to classify sentences (i.e. sequences). How can I do it ?
My attempt has been ...
1
vote
1
answer
159
views
Latent space vs Embedding space | Are they same?
I am going through variational autoencoders and it is mentioned that:
continuity (two close points in the latent space should not give two
completely different contents once decoded) and completeness ...
- 155
2
votes
2
answers
221
views
Categorical Variable Embedding
I have a categorical variable in my labeled dataset. I trained one-hot encoded version of it in another neural network having embedding layer with a larger labeled dataset. I have obtained the weights ...
- 21
0
votes
1
answer
690
views
Sequence Embedding using embedding layer: how does the network architecture influence it? [closed]
I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
0
votes
2
answers
75
views
