Skip to main content

Questions tagged [encoding]

Encoding in machine learning and data science refers to the process by which non-numeric data is transformed into a numeric representation that can be fed into machine learning algorithms.

Filter by
Sorted by
Tagged with
1 vote
1 answer
662 views

How to predict on data that is label encoded as end user will input a categorical data?

My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
0 votes
0 answers
4 views

How does encoding categorical featured with Target Encoding variants work when the Target feature is continuous?

I have been reading about Target encoding and it's variants like Leave One Out, James Stein, etc and in all cases the Target variable is itself usually binary (or can be divided into categories). How ...
0 votes
0 answers
7 views

Which method is suitable for converting categorical columns into numerical to be used in KNNImputer?

I am wondering this since the KNNImputer computes distances between rows and numerical encoding would probably skew the results unless I take a simple manhattan distance.
0 votes
1 answer
290 views

String indices must be integers

I was trying to encode the string values of the feature 'ProductCategory' into integer values, but I got this error. Also, I would like to ask if label-encoding ...
0 votes
1 answer
868 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
2 votes
2 answers
2k views

Is it vital to do label encoding with target variable

Should I always use label encoding while doing binary classification?
1 vote
1 answer
181 views

How to encode & scale IP addresses as input for ML models

Im currently working on an anomaly detection while making a transaction. As a part of the data that I extracted, I have the IP addresses of the indivduals who made the transaction. Since the IP ...
1 vote
1 answer
182 views

One Hot Encoding where all sequences don't have all values

Is there a way (other than manually creating dictionaries) to one hot encode sequences in which not all values can be present in a sequence? sklearn's OneHotEncoder ...
1 vote
1 answer
225 views

How to do target encoding when data has repeated rows?

How can I do encoding for a category when data has repeated rows? Can I do target encoding? Or Can I utilize another encoding? I want to figure out how to include a categorical variable in a model to ...
0 votes
2 answers
133 views

When to use Label Encoder and One Hot encoding with target variables?

As the title says, When to use Label Encoder and One Hot encoding with target variables ?
0 votes
0 answers
23 views

Why is positional encoding preferable over adding additional features for transformer models?

Why is information about the position not added as an additional feature? I read in forums that the only reason would be length-based overfitting, but I couldn't find a reliable source for that. What ...
0 votes
1 answer
11 views

issue with data sample

...
15 votes
3 answers
7k views

Why does frequency encoding work?

Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't ...
0 votes
1 answer
24 views

Dealing with only categorical features dataset

I'm trying to do multi-class classification on a labeled dataset with purely categorical features. There are around 30 features in total. 3 of the features in particular have around 100 unique values (...
0 votes
0 answers
19 views

What is the advantage of positional encoding over using additional features?

Popular models such as the transformer model use positional encoding on existing feature dimensions. Why is this preferred over adding more features to the feature dimension of the tensor which can ...
0 votes
1 answer
500 views

What is multi-hot encoding?

I was read and paper for machine learning, and i found this term "multi-hot encoding" without explanation. Can you help me please? the paper: https://arxiv.org/abs/2001.06917
1 vote
2 answers
1k views

Aggregating multiple encoded categorical values

I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables. I am currently using a dataset with a feature CATEGORY which has a cardinality of ~20,...
0 votes
1 answer
27 views

Encoding soft clustering results as features

I want to use cluster numbers from soft clustering algorithm output as a some sort of categorical feature (or features), add them to other features for further training in another model (Y's from soft ...
0 votes
2 answers
130 views

Handling encoding of a dataset which has more than total 2000 columns

Whenever we have a dataset to be pre processed, before feeding it to the model we convert the categorical values to numerical values for which we generally use LabelEncoding, One Hot encoding etc ...
1 vote
2 answers
490 views

How to deal with name strings in large data sets for ML?

My data set contains multiple columns with first name, last name, etc. I want to use a classifier model such as Isolation Forest later. Some word embedding techniques were used for longer text ...
2 votes
3 answers
8k views

What is the advantage of positional encoding over one hot encoding in a transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, they used positional encoding with sin for even indices and cos for odd indices. In the paper (Section 3.5), they ...
39 votes
8 answers
11k views

In a Transformer model, why does one sum positional encoding to the embedding rather than concatenate it?

While reviewing the Transformer architecture, I realized something I didn't expect, which is that : the positional encoding is summed to the word embeddings rather than concatenated to it. ...
0 votes
1 answer
1k views

splitting into train test by train_test_split of float values?

How to split into train test by train_test_split of float values ? I used LabelEncoder but I have about 300K lines and when I used the cross_val I saw ...
0 votes
0 answers
7 views

Strategies for Encoding Large Datasets in Symbolic Music Generation for BERT-type Model

I am creating a BERT-type model for symbolic music generation. An observation of my database is a musical piece. Actually, is a "viewpoint" of the piece: ...
0 votes
2 answers
119 views

Does one-hot encoding result in information loss?

BACKGROUND: I'm working with a nominal feature variable cancer_type with $5$ different classes to develop a machine learning model. One-hot encoding this feature ...
0 votes
2 answers
539 views

Handling categorical variables for Xgboost?

Currently there seems to be two approaches for handling categorical variables in gbdts: Xgboost as an option, but data need to be encoded properly (integers) Catboost can handle everything provided ...
0 votes
1 answer
34 views

Why are 1/n, 2/n, 3/n ... 2048/n not good positional encodings to be concatenated to the word vectors in transformers?

The transformer architecture has no sense of the relative positions of the word and hence we need to pass that information apriori to the along with the word embeddings to the model The positional ...
0 votes
0 answers
8 views

Best way to encode a tag column for clustering

I have a dataset which tells me a tech support case used a particular tech document. Every case has been tagged with which product it pertains to. Similarly tech documents are tagged with certain key ...
2 votes
2 answers
154 views

Explanation about i//2 in positional encoding in tensorflow tutorial about transformers

I was implementing the transformer architecture in tensorflow. I was following the tutorial : https://www.tensorflow.org/tutorials/text/transformer#setup_input_pipeline They implement the positional ...
2 votes
0 answers
65 views

Attention mechanisms without a linear layer

I am currently looking into attention mechanism as they are used in (non-Transformer) encoder-decoder architectures, meaning an architecture where some RNN (usually LSTM or GRU) is used in both the ...
3 votes
1 answer
447 views

Encode missing data and unseen data

Let's assume that I have a classification problem and all my features are categorical data. I have missing data (and I do not want to do any imputation). Also, I know that I will have some unseen ...
0 votes
1 answer
53 views

Beginner clustering project, what are the input features and how do I analyze the data?

I am a beginner to data science. I have this dataset on natural disaster events in Afghanistan from 2016 - 2017. Columns: REGION (ex. North, North West, etc) PROVINCE_NAME (kind of like US 50 states) ...
0 votes
1 answer
112 views

What's the minimum percentage of categories should be present in the categorical variable for to ignore the variable entirely

For example, if i have a feature "colour_codes" that has close to 5000 distinct color codes inside it. And the number of samples/rows is 10 million. Then should I ignore the feature "...
3 votes
2 answers
3k views

Target Encoding: missing value imputation before or after encoding

I want to perform a target encoding for my categorical features although I am not sure when to perform the data imputation if any of them has missing values. Let's say I have a few continuous features,...
1 vote
0 answers
121 views

In rotary positional embeddings (RoPE), why do we not rotate the values as well?

Actually, the question is all there is As per the paper I see that the rotations are applied only to the keys and the queries. Why are the rotations not applied to the values as well? The reasons for ...
1 vote
0 answers
144 views

CLIP Visual Transformer image encoder

I was doing some experiments with the CLIP's visual transformer encoder output (clip-ViT-B-32). So basically given the same scene or image, it should output almost ...
0 votes
1 answer
160 views

Do I need to encode samples during inference?

I recently started saving (pickling) my fitted encoders. The thinking was that I would need them to encode previously unseen samples during inference. Encode training features and labels. Train model ...
0 votes
3 answers
2k views

encoding datetime features .. label or onehot?

I am working on a cab booking prediction problem where I need to use datetime aspects like hour,day ,week etc for prediction. As I need to do categorical encoding for the purpose. can anyone help me ...
1 vote
2 answers
123 views

How to encode high cardinality categorical data?

I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are ...
2 votes
4 answers
183 views

Are scalers or encoders supposed to be serialized along with trained models?

Consider the very basic example below: ...
0 votes
1 answer
190 views

How to do encode this target vector containing strings

Consider a target vector like ...
0 votes
1 answer
22 views

Trouble Loading Lines from Text File with Various Encodings

I have been facing difficulties while loading specific lines from a text file. The lines contain characters such as ٹام بیمار ÛÛ’Û” ٹام بیمار ÛÛ’. I have tried using different ...
104 votes
4 answers
113k views

What is the positional encoding in the transformer model?

I'm trying to read and understand the paper Attention is all you need and in it, there is a picture: I don't know what positional encoding is. by listening to some youtube videos I've found out that ...
1 vote
2 answers
1k views

Can decision trees handle Nominal Categorical variables?

I have read that decision trees can handle categorical columns without encoding them. However, as decision trees make splits on the data, how does it handle Nominal Categorical variables? Surely a ...
-1 votes
1 answer
1k views

y should be a 1d array, got an array of shape (60630, 2) instead

...
0 votes
1 answer
174 views

How to treat categorical columns after ordinal encoding?

If encode three categorical variables like "bad", "normal", "good" into 0,1,2, after that can I treat them as numerical values? So can I perform on them MinMaxScaler or ...
2 votes
1 answer
40 views

What ML techniques could be used for biometric feature extraction and ID generation?

I'm working on a project that involves generating a unique ID for a given biometric (such as an iris image). I'm interested in exploring the use of ML techniques for feature extraction and ID ...
0 votes
1 answer
22 views

How do I handle "Greater than X" in a field of integers?

I've been tasked with cleaning a dataset with a "Drive Time" column that lists times taken to drive to a specific location in whole minutes. The values range from 3 to 180 minutes but there ...
1 vote
2 answers
4k views

Encoding categorical data with pre-determined dictionary

in case feature encoding, if I'd like to encode my values based on my pre-determined dictionary, how do I do that? For instance, say, I've values as ...
0 votes
1 answer
77 views

What are more advanced categorical encoding methods?

I'm familiar with the common methods: Label Encoding: {A, B, C} -> [0, 1, 2] One-Hot Encoding: ...