Skip to main content

Questions tagged [encoding]

Encoding in machine learning and data science refers to the process by which non-numeric data is transformed into a numeric representation that can be fed into machine learning algorithms.

Filter by
Sorted by
Tagged with
104 votes
4 answers
109k views

What is the positional encoding in the transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, there is a picture: I don't know what positional encoding is. by listening to some youtube videos I've found out that ...
Peyman's user avatar
  • 1,143
67 votes
2 answers
65k views

Sparse_categorical_crossentropy vs categorical_crossentropy (keras, accuracy)

Which is better for accuracy or are they the same? Of course, if you use categorical_crossentropy you use one hot encoding, and if you use ...
Master M's user avatar
  • 773
60 votes
4 answers
52k views

Difference between OrdinalEncoder and LabelEncoder

I was going through the official documentation of scikit-learn learn after going through a book on ML and came across the following thing: In the Documentation it is given about ...
Saurabh Singh's user avatar
44 votes
6 answers
40k views

Encoding features like month and hour as categorial or numeric?

Is it better to encode features like month and hour as factor or numeric in a machine learning model? On the one hand, I feel numeric encoding might be reasonable, because time is a forward ...
Funkwecker's user avatar
39 votes
8 answers
10k views

In a Transformer model, why does one sum positional encoding to the embedding rather than concatenate it?

While reviewing the Transformer architecture, I realized something I didn't expect, which is that : the positional encoding is summed to the word embeddings rather than concatenated to it. ...
FremyCompany's user avatar
27 votes
3 answers
38k views

How to deal with string labels in multi-class classification with keras?

I am newbie on machine learning and keras and now working a multi-class image classification problem using keras. The input is tagged image. After some pre-processing, the training data is represented ...
Dracarys's user avatar
  • 393
18 votes
4 answers
29k views

One hot encoding alternatives for large categorical values

I have a data frame with large categorical values over 1600 categories. Is there any way I can find alternatives so that I don't have over 1600 columns? I found this interesting link. But they are ...
vinaykva's user avatar
  • 283
16 votes
2 answers
32k views

One Hot Encoding vs Word Embedding - When to choose one or another?

A colleague of mine is having an interesting situation, he has quite a large set of possibilities for a defined categorical feature (+/- 300 different values) The usual data science approach would be ...
Jonathan DEKHTIAR's user avatar
15 votes
3 answers
7k views

Why does frequency encoding work?

Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't ...
David Masip's user avatar
  • 6,061
13 votes
1 answer
543 views

What is the difference between global and universal compression methods?

I understand that compression methods may be split into two main sets: global local The first set works regardless of the data being processed, i.e., they do not rely on any characteristic of the ...
Rubens's user avatar
  • 4,107
9 votes
3 answers
11k views

What is the difference between one-hot and dummy encoding?

I am trying to understand The reason behind encoding (one-hot encoding and dummy encoding) How one-hot and dummy are different from each other
user avatar
9 votes
2 answers
8k views

In which cases shouldn't we drop the first level of categorical variables?

Beginner in machine learning, I'm looking into the one-hot encoding concept. Unlike in statistics when you always want to drop the first level to have k-1 dummies (...
Dan Chaltiel's user avatar
9 votes
2 answers
25k views

Always drop the first column after performing One Hot Encoding?

Since one of the columns can be generated completely from the others, and hence retaining this extra column does not add any new information for the modelling process, would it be good practice to ...
Gale's user avatar
  • 403
8 votes
5 answers
6k views

How do I encode the categorical columns if there are more than 15 unique values?

I'm trying to use this data to make a data analysis report using regression. Since regression only allows for numerical types, I then need to encode the categorical data. However, most of these have ...
Cinemato's user avatar
8 votes
1 answer
13k views

Encoding with OrdinalEncoder : how to give levels as user input?

I am trying to do ordinal encoding using: from sklearn.preprocessing import OrdinalEncoder I will try to explain my problem with a simple dataset. ...
Ayush Ranjan's user avatar
7 votes
2 answers
9k views

Muti-hot encoding vs Label-Encoding

I am learning about different input-vector representations for Neural Networks One of the alternatives to sparse One-Hot encoded vector is the Multi-Hot encoding. Do I understand correctly that a ...
Kari's user avatar
  • 2,726
7 votes
2 answers
2k views

Is Label Encoding with arbitrary numbers ever useful at all?

From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here: Label Encoding - encoding a nominal ...
UchuuStranger's user avatar
6 votes
1 answer
4k views

How to handle columns with categorical data and many unique values

I have a column with categorical data with nunique 3349 values, in a 18000k row dataset, which represent cities of the world. I also have another column with 145 nunique values that I could also use ...
dungeon's user avatar
  • 175
6 votes
2 answers
4k views

One hot encoding large dataset

Initially, I have a dataset where for each row there is user_id and product_ids he bought. In that dataset there are 478191 products bought by different users. In order to discover frequent items ...
SarahData's user avatar
  • 191
6 votes
2 answers
6k views

How to use one hot encoding of string categorical features in keras?

I am dealing with a binary classification problem. The output column of my dataset is already encoded in 0/1. The problem is that I have many categorical features (columns), which are strings and I ...
ZelelB's user avatar
  • 1,057
6 votes
1 answer
289 views

Mapping of categorical features into binary indicator features

I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
Nilton Junior's user avatar
5 votes
3 answers
4k views

Words to numbers faster lookup

I'm training an LSTM for sentiment analysis on a review dataset downloaded from here. The music review dataset contains about 150K data points (reviews of varying length labelled pos or neg). After ...
Alex's user avatar
  • 767
5 votes
1 answer
625 views

On gradient boosting and types of encodings

I am having a look at this material and I have found the following statement: For this class of models [Gradient Boosting Machine algorithms] [...] it is both safe and significantly more ...
carlo_sguera's user avatar
4 votes
2 answers
2k views

is it better to correlate and encode or encode and correlate?

I have one doubt like is it better to perform label encoding and check for the correlation or should I 1st perform correlation and do label encoding? Because when I tried it both ways I'm getting ...
Nithin Reddy's user avatar
4 votes
1 answer
4k views

Is it effective to use one-hot encoding when its dimension is as large as thousands

Here I try to construct a classifier using DNN(deep neural network) with its inputs being many portfolios. In essence, each portfolio contains several stocks which are labeled by there inner-code, for ...
JunjieChen's user avatar
4 votes
1 answer
259 views

How to automate the encoding process?

I am working on the Boston challenge hosted on Kaggle and I'm still refining my features. Looking at the dataset, I realize that some columns need to be encoded in binary, some encoded in decimals (...
Andros Adrianopolos's user avatar
4 votes
1 answer
6k views

Should I do one hot encoding before feature selection and how should I perform feature selection on a dataset with both categorical and numerical data

a newbie here. I am currently self-learning data science. I am working on a dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is ...
leahnanno's user avatar
4 votes
1 answer
5k views

One hot encoding vs Word embedding

I am very confused between one hot encoding and word embedding in terms of structure of the network and how it reduces the dimensionality. I am currently using encog with c# which has some ...
Simon Nicholls's user avatar
3 votes
2 answers
4k views

Applying mean encoding before or after splitting into train and test set

I have a dataset of 50000 observations with columns of high cardinality. The best way to encode them is with mean encoding, then to use regularization. I will use CV rather than smoothing. But when I ...
Dimi's user avatar
  • 223
3 votes
1 answer
4k views

Pandas categorical variables encoding for regression (one-hot encoding vs dummy encoding)

Pandas has a method called get_dummies() that creates a dummy encoding of a categorical variable. Scikit-learn also has a OneHotEncoder that needs to be used along with a LabelEncoder. What are the ...
user90772's user avatar
  • 331
3 votes
2 answers
1k views

Choose-many categorical features: alternatives to one-hot encoding?

I'm building a model to predict the lifetime value of a client based on the relational data we have on them. The user table has a bunch of one-to-many child tables that might be predictive. Grossly ...
Autumn's user avatar
  • 133
3 votes
2 answers
1k views

How to handle non ordinal Features like Gender,Language,Region etc? Ordinal Encoding or one-hot encoding?

I see that usually, while preparing the dataset. Usually, data scientists convert non-ordinal features like Gender or Language in a dataset using LabelEncoder/ordinalEncoder. Ideally, they should have ...
Nitin Shravan's user avatar
3 votes
3 answers
2k views

One hot encoding with too many features (~ 10,000)

I am building a model to predict time off and sick leave for a specific employee. Each of the employees has one row per day from 01/01/2013 to 31/12/2018 in the dataset flagged with 0 or 1 (if that ...
Aggamarcel's user avatar
3 votes
1 answer
2k views

How do I specify encoding in scikit-learn OrdinalEncoder?

Scikit-learn object OrdinalEncoder() allows the user to create a lineary based encoding principle for ordinal data, however the the codes are encoded randomly. Is there any way I can specify how the ...
Sam's user avatar
  • 31
3 votes
2 answers
411 views

why One-Hot Encoder can avoid the situation that the model will misunderstand the data to be in some kind of order if the data has been Label Encoding

We know that we prefer to using One-Hot Encoding not Label Encoding when processing with non-ordinal data. And I real a blog which give the difference between Label Encoding and One-Hot Encoding. So ...
Bowen Peng's user avatar
3 votes
1 answer
157 views

Aggregating target-encoded array-like categorical features?

I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables for machine learning classification algorithms. One-hot encoding leads to very high ...
user4446237's user avatar
3 votes
2 answers
3k views

Target Encoding: missing value imputation before or after encoding

I want to perform a target encoding for my categorical features although I am not sure when to perform the data imputation if any of them has missing values. Let's say I have a few continuous features,...
MarkSt's user avatar
  • 31
3 votes
1 answer
3k views

One-hot encode multi-class multi-label sequences

Suppose I want to build a timeseries where each timestep is represented by a categorical array: the encoded sequences look like [[2, 0, 5],[3, 1, 4],..] and each ...
ginevracoal's user avatar
3 votes
1 answer
534 views

"Binary Encoding" in "Decision Tree" / "Random Forest" Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170,000 rows One of the categoric variables has ...
Ahmet's user avatar
  • 31
3 votes
1 answer
441 views

Encode missing data and unseen data

Let's assume that I have a classification problem and all my features are categorical data. I have missing data (and I do not want to do any imputation). Also, I know that I will have some unseen ...
Outcast's user avatar
  • 1,057
3 votes
1 answer
1k views

For a multi-class classification problem, how to transform the target variable to a form that is usable by sklearn algorithms? [duplicate]

I recently tried to create a model for predicting what class a sample belongs to out of 160 possible classes. These classes of the target variable are just simple strings describing workouts like "...
austinkjensen's user avatar
3 votes
1 answer
159 views

How to work with different Encoding for Foreign Languages

I've got a Word Embedding File called model.txt. This contains 100 Dimensional vectors for over a million French words. These words contain accented characters such ...
MetaInformation's user avatar
3 votes
0 answers
774 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
Blenz's user avatar
  • 2,074
2 votes
3 answers
2k views

Does One-Hot encoding increase the dimensionality and sparsity of dataset?

There are two ways to convert object datatype into numeric datatype, first is One-Hot encoding and second is simply map the numerical tags to different values. For example for column Age containing ...
Ahmad Bilal's user avatar
2 votes
3 answers
7k views

What is the advantage of positional encoding over one hot encoding in a transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, they used positional encoding with sin for even indices and cos for odd indices. In the paper (Section 3.5), they ...
Inderpartap Cheema's user avatar
2 votes
1 answer
587 views

Do categorical features always need to be encoded?

I'm using Spark's Machine Learning Library, and features are categorical. The features are strings, and Spark's MLlib (like many other machine learning libraries) does not accept Strings as inputs. ...
gbhrea's user avatar
  • 307
2 votes
1 answer
5k views

One hot encoding at character level with Keras

I am reading Chollet's book on deep learning at the moment and in the NLP chapter he says: ...
Learning is a mess's user avatar
2 votes
1 answer
80 views

Encode a set of skills into a feature

I am working with a dataset where users have a set of skills. I have more than 500 skills and I was wondering what is the best way of encoding a vector, e.g., ...
vftw's user avatar
  • 143
2 votes
3 answers
2k views

Transformation of categorical variables (binary vs numerical)

When using categorical encoding, I see some authors use arbitrary numerical transformation while others use binary transformation. For example, if I have a feature vector with values A, B and c. The ...
U. User's user avatar
  • 257
2 votes
1 answer
409 views

Encoding for Linear Regression

I have a CSV file with salary information and other columns. I am trying to transform some of these columns into proper values, for a ...
Alix Blaine's user avatar