Questions tagged [encoder]
The encoder tag has no usage guidance.
27
questions
0
votes
0
answers
7
views
Low accuracy during inference for Sbert Cross Encoder
I have an sbert Cross-Encoder model to evaluate sentence pairs. The first sentence is a ticket description and the second is the description of category it falls into. The test accuracy is ~90 p.c on ...
- 1
0
votes
1
answer
62
views
Keras MLP not working
For my degree's final project I am working with Keras and trying to build different AI models. I'm having trouble with an MLP. First I preprocess the UNSW-NB15 dataset and later use it as input in a ...
0
votes
1
answer
58
views
How is RNN decoder output calculated?
I was trying to read RNN Encoder Decoder paper.
RNN (plain RNN i.e. non encoder-decoder RNN)
It starts with giving equation for RNN:
hidden state in RNN is given as:
... equation (1)
where f is a ...
- 179
1
vote
2
answers
790
views
What does the output of an encoder in encoder-decoder model represent?
So in most blogs or books touching upon the topic of encoder-decoder architectures the authors usually say that the last hidden state(s) of the encoder is passed as input to the decoder and the ...
- 63
0
votes
0
answers
24
views
Explain the outputs of Bidirectional LSTM . There are 5 output , which of them are hidden state and cell state respectively?
I am trying to use bidirectional LSTM as encoder in my translation model.
I set return_sequence=True and return_state=True and I ...
- 135
0
votes
0
answers
21
views
Why we need encoder-decoder architectures despite we already have RNN?
Why we need encoder-decoder architectures despite we already have RNN? From Googling, I was just told such architecture is used, in the context of NLP, that it allows:
The key benefits of the ...
- 411
0
votes
0
answers
88
views
Use Beam Search in encoder-decoder
How can tfa.seq2seq.BeamSearchDecoder, for example, be used with a simple encoder-decoder architecture? Suppose the task is machine translation, where the encoder returns a vector representation of ...
1
vote
1
answer
188
views
Encode categorical data for unsupervised learning
What is the best encoder for categorical data in unsupervised learning?
I am using unsupervised learning on mixed data (such as K-means).
Before running my unsupervised algorithm, I am using dimension ...
- 11
1
vote
0
answers
33
views
Motivation of LSTM with no Input
I have read this paper where authors use LSTM to learn the attention applied to several sets. They use LSTM without input or output, LSTM just uses the hidden state and evolves it:
My question is ...
- 11
0
votes
1
answer
57
views
SkLearn Categorical Naive Bayes Vs Mathematical theory of Naive Bayes
The Naive Bayes classification based on the following formula
$P(C_i|X) = {P(X|C_i)P(C_i) \over P(X)} ... i)$
$P(X|C_i)$ is the posterior probability of $X$ conditioned on $C_i$, $P(X)$ prior ...
- 359
0
votes
1
answer
51
views
Encoder Decoder model for parameter extraction from text input
I have an input as text from which I want to extract parameters as given in example below.
Input:
...
- 101
0
votes
1
answer
42
views
Comparing encoders to same input of differnt output size
Let's say I have an input s1 and I pass it to two encoders e1 and e2. They output encodings ...
2
votes
1
answer
2k
views
What to do with Transformer Encoder output?
I'm in the middle of learning about Transformer layers, and I feel like I've got enough of the general idea behind them to be dangerous. I'm designing a neural network and my team would like to ...
- 23
2
votes
2
answers
1k
views
Is it vital to do label encoding with target variable
Should I always use label encoding while doing binary classification?
- 21
1
vote
1
answer
1k
views
How to Visualize attention weights in a Attention based Encoder-Decoder network in Time series forecasting
Below is one example Attention-based Encoder-decoder network for multivariate time series forecasting task. I want to visualize the attention weights.
...
- 113
1
vote
1
answer
469
views
Get Hidden Layers in PyTorch TransformerEncoder
I am trying to access the hidden layers when using TransformerEncoder and TransformerEncoderLayer. I could not find anything ...
- 13
0
votes
1
answer
27
views
Can anyone interpret this Recurrent Network Encoder-Decoder question?
I'm trying to earn some extra credit, so the professor won't elaborate further on what's being asked in this question:
The dataset that we're given is a line-by-line file of protein sequences (...
0
votes
1
answer
2k
views
Changing order of LabelEncoder() result
Assume I have a multi-class classification task. The labels are:
Class 1
Class 2
Class 3
After LabelEncoder(), the labels are transformed into 0-1-2.
My questions are:
Do the labels have to start ...
- 175
0
votes
1
answer
284
views
Encoder-Decoder LSTM for Trajectory Prediction
I need to use encoder-decoder structure to predict 2D trajectories. As almost all available tutorials are related to NLP -with sparse vectors-, I couldn't be sure about how to adapt the solutions to a ...
1
vote
1
answer
404
views
Encode time-series of different lengths with keras
I have time-series as my data (one time-series per training example). I would like to encode the data within these series in a fixed-length vector of features using a keras model.
The problem is that ...
- 181
3
votes
2
answers
2k
views
What is the difference between BERT architecture and vanilla Transformer architecture
I'm doing some research for the summarization task and found out BERT is derived from the Transformer model. In every blog about BERT that I have read, they focus on explaining what is a bidirectional ...
- 133
3
votes
1
answer
3k
views
Why transform embedding dimension in sin-cos positional encoding?
Positional encoding using sine-cosine functions is often used in transformer models.
Assume that $X \in R^{l\times d}$ is the embedding of an example, where $l$ is the sequence length and $d$ is the ...
- 165
2
votes
2
answers
2k
views
Role of decoder in Transformer?
I understand the mechanics of Encoder-Decoder architecture used in the Attention Is All You Need paper. My question is more high level about the role of the decoder. Say we have a sentence translation ...
- 165
1
vote
1
answer
1k
views
How to add a Decoder & Attention Layer to Bidirectional Encoder with tensorflow 2.0
I am a beginner in machine learning and I'm trying to create a spelling correction model that spell checks for a small amount of vocab (approximately 1000 phrases). Currently, I am refering to the ...
- 11
1
vote
1
answer
1k
views
sklearn serialize label encoder for multiple categorical columns
I have a model with several categorical features that need to be converted to numeric format. I am using a combination of LabelEncoder and OneHotEncoder to achieve this.
Once in production, I need to ...
- 133
2
votes
1
answer
104
views
Encoding correlation
I have rather theory-based question as I'm not that experienced in encoders, embeddings etc. Scientifically I'm mostly oriented around novel evolutionary model-based methods.
Let's assume we have ...
- 814
1
vote
1
answer
351
views
How do I implement Dual-encoder model in Pytorch?
I am trying to implement the paper titled Learning Cross-lingual Sentence Representations via a Multi-task Dual-Encoder Model.
Here the encoder and decoder share the same weights but I am unable to ...
- 341