Skip to main content

Questions tagged [encoding]

Encoding in machine learning and data science refers to the process by which non-numeric data is transformed into a numeric representation that can be fed into machine learning algorithms.

Filter by
Sorted by
Tagged with
15 votes
3 answers
7k views

Why does frequency encoding work?

Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't ...
David Masip's user avatar
  • 6,061
7 votes
2 answers
2k views

Is Label Encoding with arbitrary numbers ever useful at all?

From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here: Label Encoding - encoding a nominal ...
UchuuStranger's user avatar
44 votes
6 answers
40k views

Encoding features like month and hour as categorial or numeric?

Is it better to encode features like month and hour as factor or numeric in a machine learning model? On the one hand, I feel numeric encoding might be reasonable, because time is a forward ...
Funkwecker's user avatar
6 votes
1 answer
289 views

Mapping of categorical features into binary indicator features

I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
Nilton Junior's user avatar
2 votes
1 answer
2k views

String handling by OneHotEncoder

I am reading everywhere on new questions and blogs that since version 0.20, OneHotEncoder is able to handle string features. Moreover, the documentation is what looks more ambiguous. Here are the ...
pragun's user avatar
  • 171
104 votes
4 answers
109k views

What is the positional encoding in the transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, there is a picture: I don't know what positional encoding is. by listening to some youtube videos I've found out that ...
Peyman's user avatar
  • 1,143
60 votes
4 answers
52k views

Difference between OrdinalEncoder and LabelEncoder

I was going through the official documentation of scikit-learn learn after going through a book on ML and came across the following thing: In the Documentation it is given about ...
Saurabh Singh's user avatar
39 votes
8 answers
10k views

In a Transformer model, why does one sum positional encoding to the embedding rather than concatenate it?

While reviewing the Transformer architecture, I realized something I didn't expect, which is that : the positional encoding is summed to the word embeddings rather than concatenated to it. ...
FremyCompany's user avatar
9 votes
2 answers
8k views

In which cases shouldn't we drop the first level of categorical variables?

Beginner in machine learning, I'm looking into the one-hot encoding concept. Unlike in statistics when you always want to drop the first level to have k-1 dummies (...
Dan Chaltiel's user avatar
7 votes
2 answers
9k views

Muti-hot encoding vs Label-Encoding

I am learning about different input-vector representations for Neural Networks One of the alternatives to sparse One-Hot encoded vector is the Multi-Hot encoding. Do I understand correctly that a ...
Kari's user avatar
  • 2,726
4 votes
1 answer
4k views

Is it effective to use one-hot encoding when its dimension is as large as thousands

Here I try to construct a classifier using DNN(deep neural network) with its inputs being many portfolios. In essence, each portfolio contains several stocks which are labeled by there inner-code, for ...
JunjieChen's user avatar
3 votes
2 answers
1k views

How to handle non ordinal Features like Gender,Language,Region etc? Ordinal Encoding or one-hot encoding?

I see that usually, while preparing the dataset. Usually, data scientists convert non-ordinal features like Gender or Language in a dataset using LabelEncoder/ordinalEncoder. Ideally, they should have ...
Nitin Shravan's user avatar
2 votes
3 answers
2k views

Transformation of categorical variables (binary vs numerical)

When using categorical encoding, I see some authors use arbitrary numerical transformation while others use binary transformation. For example, if I have a feature vector with values A, B and c. The ...
U. User's user avatar
  • 257
2 votes
3 answers
7k views

What is the advantage of positional encoding over one hot encoding in a transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, they used positional encoding with sin for even indices and cos for odd indices. In the paper (Section 3.5), they ...
Inderpartap Cheema's user avatar
1 vote
2 answers
979 views

Unsupervised encoding of categorical features

I have multiple log records with discrete categorical features. Shape of my dataset is (100k, 24) My aim is to look for anomalies in these records. I am planning to cluster the data after encoding. ...
Raghav Kukreti's user avatar
0 votes
2 answers
3k views

Binary Classification - One Hot Encoding preventing me using Test Set [duplicate]

I have a preprocessing pipeline that includes replacing missing values and onehotencoding for the categorical variables. When I try to use my model on the test set, it explains that the number of ...
Viraj Vaitha's user avatar