Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

For a fine-tuning a transformer to type like a specific person, should I use sentence semantic embeddings or word semantic embeddings

I'm not clear on the pros and cons of each one for this particular task. Is there even a meaningful difference? My guess is using semantic embeddings for words will be better in nearly all cases ...
Austin Capobianco's user avatar
0 votes
0 answers
10 views

Handling Irregular Time Series Data with Multiple Categories in Hourly Resolution Embeddings

I'm working with a time series dataset that has 100 categories for a single feature, and I've generated embeddings for these categories. However, my problem arises when I am trying to sample the data ...
Mohammad Asif Ibna Mustafa's user avatar
0 votes
0 answers
27 views

Is openAI text generation models an extension of embedding models?

we can creating embeddings using below code ...
Vinay Sharma's user avatar
0 votes
0 answers
27 views

Using Embedding For Regularization

Is using embeddings for regularization a valid practice? My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
Adenilson Arcanjo's user avatar
1 vote
1 answer
59 views

Why is T5 often used in text-to-data for text prompt encoders?

In the text-to-data(music, image, audio, etc.) generative AI field, one method of encoding text prompts is to use pre-trained language models. Such an approach was used in research on Moûsai [1] and ...
NakataKoo's user avatar
0 votes
0 answers
29 views

Combining Textual, Categorical and Numerical data for Semantic Search using SentenceTransformers model

I'm building a food semantic search model and I want to use a pre-trained SentenceTransformers model with cosine similarity. I'm using Epicurious dataset for the corpus which consists of textual (&...
Alex's user avatar
  • 1
0 votes
1 answer
25 views

Using text embeddings directly to compute similarity vs using them as features for a model that predicts similariy

Say you have a problem where you have a query and a set of result documents and you want to rank the result documents according to the query. Say also you have embeddings for the query and for the ...
user1893354's user avatar
2 votes
0 answers
48 views

CLIP Visual Transformer image encoder

I was doing some experiments with the CLIP's visual transformer encoder output (clip-ViT-B-32). So basically given the same scene or image, it should output almost ...
Tina J's user avatar
  • 217
0 votes
1 answer
57 views

Are the terms "vector embedding" and "embedding vector" interchangeable?

Googling "vector embedding" gives me ~40k results, including from Pinecone, Weaviate, and OpenAI. Googling ...
Alex Hall's user avatar
  • 103
0 votes
0 answers
22 views

Are Vector embeddings position or direction vectors?

I've seen statements about embeddings being positions on a hypersphere. I've also seen statements about using dot product to determine similarity of these positions. This implies to me that we care ...
iasksillyquestions's user avatar
0 votes
0 answers
621 views

How to add knowledge to the LLM using LangChain (at a high level)?

At a super high level, I would like to create a fantasy language AI tutor. For this question, however, I would like to better understand how, generally speaking, you add your own custom data/knowledge ...
Lance's user avatar
  • 263
2 votes
2 answers
301 views

Why cant we use normalise position encodings instead of the cos and sine encodings used in the Transformer paper?

I'm working with Transformer models for sequence-to-sequence tasks and I'm trying to fully understand the use of positional encodings in these models. In the original "Attention is All You Need&...
mutli-arm-bandit's user avatar
0 votes
2 answers
570 views

Recommended way to embed a text thousands of tokens long?

I've split the text up sections each 512 tokens long and created embeddings for each of them. I want to combine them into 1 embedding for the full text. How do I do that? Is this even recommended? ...
codeananda's user avatar
0 votes
0 answers
64 views

Why are similarities all close to 1 in my two tower recommender?

I am trying to build a simple two tower recommender system on the MovieLens 100k dataset. The user tower is just a simple embedding layer. The item tower uses an embedding layer and concats that with ...
ds_'s user avatar
  • 1
0 votes
1 answer
28 views

How do I interpret low dimentional embeddings of high dimentional embeddings?

I am trying to understand what I am supposed to learn about a problem when using dimensionality reduction methods. In particular, I am referring to methods like t-SNE and UMAP. For the most part I am ...
Finncent Price's user avatar
0 votes
0 answers
14 views

High-Cardinality Categorical Feature with frequency score

I'm working with a really high-cardinality feature as one of the inputs to my model and I'm using hash-encoded feature embedding rather than one-hot encoding. However, this method is ignoring the ...
UK-07's user avatar
  • 101
1 vote
0 answers
118 views

Otimization of similarity search for multiple embeddings by creating a weighted artificial embedding

I have embeddings of text created with a BERT model. A group of these embeddings should be used to find similar embeddings corresponding to this group. I know that you can use average or max (or ...
soph's user avatar
  • 113
0 votes
0 answers
30 views

Max word embedding

I'm trying to understand the concept behind Max Word / Sentence embeddings also in contrast to mean word embedding. They used it for example in this article. So far I didn't find good resources which ...
soph's user avatar
  • 113
0 votes
0 answers
43 views

Distance Metric for a dataset with embeddings and numerical columns

I'm trying to build an Approximate Nearest Neighbours model that can fetch similar records in a dataset, that are contextually similar. For example, for a record of job=Software Engineer and age=25, a ...
Augustine Samuel's user avatar
0 votes
0 answers
17 views

Align sub-sentences with sentences using embeddings

I would like ask for ideas So I have sub-sentences, embedded in the context of their respective full-sentences. Then, I have other full-sentences and I would like to find a) if they have similar sub-...
aqua's user avatar
  • 123
0 votes
0 answers
9 views

How to generate embedding for Image Dataset with multiple features?

My dataset has Jewellery images. The jewellery is of 4 types: Type Color Shape Material Now I am aware of converting images to embeddings. I need to group similar images like two plain gold rings ...
Lakshay Dulani's user avatar
0 votes
0 answers
21 views

How the embedding model (x-vectors) trained?

I read this paper: X-Vectors: Robust DNN Embeddings for Speaker Recognition which describes how PyAnnote embedding block works. I'm not sure I understand how the X-Vector model was trained and tested: ...
user3668129's user avatar
0 votes
1 answer
210 views

SBERT Embeddings from Conversations

I have a dataset consisting of text-based conversations between two humans. One conversation has on average 20 turns and can look as follows: ...
DictionaryProver's user avatar
0 votes
0 answers
11 views

In contrastive loss, why do we sample negatives from the document, not the query?

From what I read, for image search contrastive training, we give one query in text, and 1 positive image, and (N-1) sampled negative images. However, for video search contrastive training, we give 1 ...
Herbert's user avatar
  • 101
0 votes
0 answers
19 views

Does word2vec skip gram involves softmax in the output layer

I was going through various pytorch and from-scratch implementations of skip-gram. I found following: This implementaiton does not seem to use softmax ...
RajS's user avatar
  • 103
1 vote
1 answer
41 views

Visualizing convolutional neural networks embedding

In this article, the author creates a graph (at the end of the post) from the embeddings of different words found by transformer model. I would like to do a similar thing for a convolutional neural ...
Zan's user avatar
  • 113
0 votes
0 answers
36 views

Inductive embedding technique for nodes without features

I am working on a project that involves graph embeddings. I have an homogeneous directed graph where the nodes have no features. The only available information are the edges between nodes. I need to ...
Frank's user avatar
  • 1
0 votes
1 answer
29 views

Is there a way to embed curves into smaller vectors?

I know about embeddings for words, but I would like to know if it is possible to do something similar for curves. What I mean by curves is a curve of a function. Say I have 1000 points corresponding ...
WLD's user avatar
  • 1
0 votes
0 answers
24 views

Select Topic Words from Clusters

I am following this solution for clustering: https://towardsdatascience.com/clustering-contextual-embeddings-for-topic-model-1fb15c45b1bd For step four "Select Topic Words from Clusters", I ...
SaNa's user avatar
  • 129
1 vote
1 answer
39 views

Which pre-trained model to select to generate embeddings from shop names written in English?

Good afternoon! I have a dataset with thousands of shop names written in English. Several shop names might belong to one business entity, for instance, shops with names "KFC 001", "WWW....
rsx's user avatar
  • 11
0 votes
0 answers
32 views

Embedding: Can i use it in a time series problem?

I'm trying to do feature extraction in some discretized time series with a variable length, doing that i'm creating an RNN auto encoder. My main problem is to find a way to let the model train with ...
Nathaldien's user avatar
0 votes
1 answer
21 views

Are there methods to represent entire knowledge graphs using a single embedding vector?

In a knowledge graph, embedding vectors can be learned for nodes (node embedding) and edges (edge embeddings). Is there a method to learn one single embedding vector for the entire knowledge graph?
skumaravel's user avatar
1 vote
0 answers
68 views

Machine learning with mixed variables dataset (numerical, categorical and embeddings)

I'm working on a machine learning project where I'm trying to predict the revenue of a movie. My dataset contains mixed data types. There are numerical features (rating, number of votes, release year,....
Mathieu Rousseau's user avatar
1 vote
0 answers
110 views

Embedding performing unexpectedly worse than multi-hot encoding

I'm pretty new to ML and have set up a Keras model with a number of categorical features of users & items to predict positive reactions. Each feature is a list of IDs from a database - i.e. ...
Tom's user avatar
  • 11
0 votes
0 answers
54 views

Practical application of denoising autoencoders

I have been reading into autoencoders for the purpose of denoising data. In the examples i found (eg. [1, 2, 3], which are the first few google results) they have the following input/output: Input ...
Leander Moesinger's user avatar
1 vote
1 answer
54 views

Understand the reason of embedding and the size inside it in Pytorch

I'm very new to pytorch - taking a course in udemy. There is something I find hard to understand and would like to get explaination about, in a bit simpler words than what I can find in the ...
Reut's user avatar
  • 349
1 vote
1 answer
63 views

Biometrics identification with embeddings comparison and "unknown"/"other" class/label

This is a general or more conceptual questions about biometric classification models, based on deep learning neural networks. The goal of the system is to take a set of features (e.g. voice recording, ...
Triceratops's user avatar
1 vote
1 answer
79 views

Product embeddings

Hei, I have a list of purchase baskets from customers and would like to build embeddings for the products. For example: BASKET1 = ['PRODUCT234', 'PRODUCT214', 'PRODUCT768'] BASKET2 = ['PRODUCT2', '...
ryuzakinho's user avatar
1 vote
1 answer
139 views

Is there a sensible notion of 'character embeddings'?

There are several popular word embeddings available (e.g., Fasttext and GloVe); In short, those embeddings are a tool to encode words along with a sensible notion of semantics attached to those words (...
Ramiro Hum-Sah's user avatar
1 vote
0 answers
242 views

How are the embedding and context matrices created and updated in word embedding?

I am struggling to understand how word embedding works, especially how the embedding matrix $W$ and context matrix $W'$ are created/updated. I understand that in the Input we may have a one-hot ...
Revolucion for Monica's user avatar
2 votes
1 answer
2k views

Transformer time series classification using time2vec positional embedding

I want to use a transformer model to do classification of fixed-length time series. I was following along this tutorial using keras which uses time2vec as a positional embedding. According to the ...
Reignbeaux's user avatar
1 vote
1 answer
647 views

Do we perform text embedding before or after train-test splitting?

Do we perform text embedding before or after train-test splitting? I know that for encoding variables, usually done after the split. However, I'm not sure if that's also the case for text processing?
Student's user avatar
  • 411
2 votes
0 answers
16 views

Dicussion for X-vectors

I am posting this question to ask some questions regarding the X-vector embedding proposed by Synder et al. The paper is X-VECTORS: ROBUST DNN EMBEDDINGS FOR SPEAKER RECOGNITION With reference to the ...
Leo's user avatar
  • 125
1 vote
1 answer
473 views

What is the meaning of two embedding layers in a row?

I've noticed in one deep pre-trained textual neural network that there are two embedding layers in the beginning and I don't quite understand why there are two of them. As far as I understand (correct ...
Igor Igor's user avatar
  • 157
1 vote
0 answers
76 views

Graph Neural Network | How node embeddings are learned from several graphs?

I am reading paper on MEGnet which is a GNN. The objective is that we have several molecules that share same elements such as molecules $C0_2$ and $COOH$ share $C$ and $O$. Now if we learn the node ...
user0193's user avatar
  • 155
1 vote
1 answer
2k views

How can I get the output of a Keras LSTM layer?

I want to get the output (that is a vector) of a LSTM layer of a network built in Python using Keras and that is trained to classify sentences (i.e. sequences). How can I do it ? My attempt has been ...
HelpNeederStudent's user avatar
1 vote
1 answer
159 views

Latent space vs Embedding space | Are they same?

I am going through variational autoencoders and it is mentioned that: continuity (two close points in the latent space should not give two completely different contents once decoded) and completeness ...
user0193's user avatar
  • 155
2 votes
2 answers
221 views

Categorical Variable Embedding

I have a categorical variable in my labeled dataset. I trained one-hot encoded version of it in another neural network having embedding layer with a larger labeled dataset. I have obtained the weights ...
Agile's user avatar
  • 21
0 votes
1 answer
690 views

Sequence Embedding using embedding layer: how does the network architecture influence it? [closed]

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there ...
HelpNeederStudent's user avatar
0 votes
2 answers
75 views

zero padding problem [closed]

i need to implement this code by using padding ...
Begnnier's user avatar