Questions tagged [encoder]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
24 views

SkLearn Categorical Naive Bayes Vs Mathematical theory of Naive Bayes

The Naive Bayes classification based on the following formula $P(C_i|X) = {P(X|C_i)P(C_i) \over P(X)} ... i)$ $P(X|C_i)$ is the posterior probability of $X$ conditioned on $C_i$, $P(X)$ prior ...
  • 309
0 votes
0 answers
17 views

Self-Attention: Tensorflow to PyTorch Code Conversion

I have a TensorFlow encoder code with a Self-attention layer X = layers.Attention()([X,X]) #Single-head Self-attention layer What is the equivalent PyTorch code? ...
  • 101
0 votes
1 answer
19 views

Encoder Decoder model for parameter extraction from text input

I have an input as text from which I want to extract parameters as given in example below. Input: ...
0 votes
1 answer
14 views

Comparing encoders to same input of differnt output size

Let's say I have an input s1 and I pass it to two encoders e1 and e2. They output encodings ...
0 votes
1 answer
48 views

What to do with Transformer Encoder output?

I'm in the middle of learning about Transformer layers, and I feel like I've got enough of the general idea behind them to be dangerous. I'm designing a neural network and my team would like to ...
  • 3
1 vote
1 answer
210 views

Is it vital to do label encoding with target variable

Should I always use label encoding while doing binary classification?
1 vote
1 answer
237 views

How to Visualize attention weights in a Attention based Encoder-Decoder network in Time series forecasting

Below is one example Attention-based Encoder-decoder network for multivariate time series forecasting task. I want to visualize the attention weights. ...
0 votes
0 answers
108 views

Special tokens for encoder and decoder in the transformer architecture

I am trying to wrap my head around the different special tokens that the different transformer architectures use. For example, let's say we have the following input and target both for a text ...
0 votes
0 answers
8 views

Natural Language gender classification task with very small training set

The task involving determining the gender of the creator of a Reddit post. Given a post and its title, I need a model to output a probability vector $[p_{male},p_{female}]$. The difficulty here is ...
1 vote
1 answer
103 views

Get Hidden Layers in PyTorch TransformerEncoder

I am trying to access the hidden layers when using TransformerEncoder and TransformerEncoderLayer. I could not find anything ...
0 votes
1 answer
24 views

Can anyone interpret this Recurrent Network Encoder-Decoder question?

I'm trying to earn some extra credit, so the professor won't elaborate further on what's being asked in this question: The dataset that we're given is a line-by-line file of protein sequences (...
0 votes
1 answer
1k views

Changing order of LabelEncoder() result

Assume I have a multi-class classification task. The labels are: Class 1 Class 2 Class 3 After LabelEncoder(), the labels are transformed into 0-1-2. My questions are: Do the labels have to start ...
  • 175
0 votes
1 answer
220 views

Encoder-Decoder LSTM for Trajectory Prediction

I need to use encoder-decoder structure to predict 2D trajectories. As almost all available tutorials are related to NLP -with sparse vectors-, I couldn't be sure about how to adapt the solutions to a ...
0 votes
1 answer
198 views

Encode time-series of different lengths with keras

I have time-series as my data (one time-series per training example). I would like to encode the data within these series in a fixed-length vector of features using a keras model. The problem is that ...
3 votes
1 answer
1k views

What is the difference between BERT architecture and vanilla Transformer architecture

I'm doing some research for the summarization task and found out BERT is derived from the Transformer model. In every blog about BERT that I have read, they focus on explaining what is a bidirectional ...
2 votes
1 answer
1k views

Why transform embedding dimension in sin-cos positional encoding?

Positional encoding using sine-cosine functions is often used in transformer models. Assume that $X \in R^{l\times d}$ is the embedding of an example, where $l$ is the sequence length and $d$ is the ...
  • 55
2 votes
2 answers
839 views

Role of decoder in Transformer?

I understand the mechanics of Encoder-Decoder architecture used in the Attention Is All You Need paper. My question is more high level about the role of the decoder. Say we have a sentence translation ...
  • 55
1 vote
1 answer
1k views

How to add a Decoder & Attention Layer to Bidirectional Encoder with tensorflow 2.0

I am a beginner in machine learning and I'm trying to create a spelling correction model that spell checks for a small amount of vocab (approximately 1000 phrases). Currently, I am refering to the ...
  • 11
1 vote
1 answer
636 views

sklearn serialize label encoder for multiple categorical columns

I have a model with several categorical features that need to be converted to numeric format. I am using a combination of LabelEncoder and OneHotEncoder to achieve this. Once in production, I need to ...
  • 123
2 votes
1 answer
75 views

Encoding correlation

I have rather theory-based question as I'm not that experienced in encoders, embeddings etc. Scientifically I'm mostly oriented around novel evolutionary model-based methods. Let's assume we have ...
1 vote
1 answer
249 views

How do I implement Dual-encoder model in Pytorch?

I am trying to implement the paper titled Learning Cross-lingual Sentence Representations via a Multi-task Dual-Encoder Model. Here the encoder and decoder share the same weights but I am unable to ...
  • 341