Questions tagged [encoding]

The tag has no usage guidance.

35 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
416 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
2
votes
1answer
39 views

How can I count the number of occurrences of a category in dataset as part of an Sklearn Pipeline

Let us say we have a dataset with a feature such as Surname. arr['Surname'] = ['Smith', 'Jones', 'Johnson', 'Smith'] And I want to encode this categorical info ...
2
votes
0answers
48 views

Why don't Target/LeaveOneOut Encoders work well for Regression Tasks?

In this review of categorical encoding, it states early on that For regression tasks, Target and LeaveOneOut probably won’t work well and later repeats that Target/LeaveOneOut (Owen Zhang's ...
2
votes
0answers
113 views

how to extract the Top contributing labels/words in universal-sentence-encoder-large - TransformerModel?

I'm using the universal-sentence-encoder-large (Transformer Model) encoding process for embedding and then using the embedding for Clustering - Basically for unsupervised learning. I want to get the ...
2
votes
4answers
70 views

Equivalent of numeric encoding when rows can contain multiple values

If we have a column like: Name 0 Alice 1 Bob 2 Dave then, after numeric encoding, it becomes: ...
2
votes
0answers
142 views

“Binary Encoding” in “Decision Tree” / “Random Forest” Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170000 rows One of the categoric variables has ...
1
vote
1answer
13 views

Explanation about i//2 in positional encoding in tensorflow tutorial about transformers

I was implementing the transformer architecture in tensorflow. I was following the tutorial : https://www.tensorflow.org/tutorials/text/transformer#setup_input_pipeline They implement the positional ...
1
vote
0answers
119 views

Encoding with OrdinalEncoder : how to give levels as user input?

I am trying to do ordinal encoding using: from sklearn.preprocessing import OrdinalEncoder I will try to explain my problem with a simple dataset. ...
1
vote
1answer
78 views

Encode missing data and unseen data

Let's assume that I have a classification problem and all my features are categorical data. I have missing data (and I do not want to do any imputation). Also, I know that I will have some unseen ...
1
vote
0answers
52 views

Aggregating target-encoded array-like categorical features?

I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables for machine learning classification algorithms. One-hot encoding leads to very high ...
1
vote
0answers
113 views

How to encode H3 geohash in regression model

I'm trying to train a random forest regression model based on a number of features, including location. I know that raw lat/long can't be used directly, so I've bucketed them using H3. I'm struggling ...
1
vote
0answers
36 views

How to pre-process frequency of a series of signals?

I use a neural net to generate predictions based on a time series of signals. I use a sliding window to feed the data to an LSTM model. The input signals have a random frequency that - I believe - is ...
0
votes
3answers
48 views

How do i get dummies for this dataset

I am using an udemy course for MachineLearning and I am trying to form a dummy for my variable the column is Country I want to change to France Germany Spain France ...
0
votes
0answers
63 views

What is “position” in CNN (im2latex) for Positional Encoding?

I'm trying to build a model that maps images of math formulas into LaTeX markup. I found an acticle (https://arxiv.org/ftp/arxiv/papers/1908/1908.11415.pdf) that proposes an encoder-decoder ...
0
votes
0answers
14 views

Target Encoding and Feature Scaling

I am using Support Vector Classification which performs well when we have done Feature Scaling, however, I am using Target Encoding on my categorical variables. Is it advisable to do Feature Scaling ...
0
votes
0answers
94 views

Mean encoding With KFold regularization

I just learned that regularizing the mean encoding reduce the leakage hence generalize better than mean encoding without it but I made 2 submissions with XGB in <...
0
votes
1answer
57 views

Aggregating multiple encoded categorical values

I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables. I am currently using a dataset with a feature CATEGORY which has a cardinality of ~20,...
0
votes
1answer
37 views

How to do target encoding when data has repeated rows?

How can I do encoding for a category when data has repeated rows? Can I do target encoding? Or Is there another encoding I can use? I want to figure how to include a categorical variable in a model ...
0
votes
0answers
29 views

One Hot Encoding where all sequences don't have all values

Is there a way (other than manually creating dictionaries) to one hot encode sequences in which not all values can be present in a sequence? sklearn's OneHotEncoder ...
0
votes
0answers
9 views

Can i use different types of encodings for categorical variables in one dataset

Should I mix encodings. For example for features age and income i have one type of encoding and for features typeOfPerson i have another?
0
votes
1answer
183 views

splitting into train test by train_test_split of float values?

How to split into train test by train_test_split of float values ? I used LabelEncoder but I have about 300K lines and when I used the cross_val I saw ...
0
votes
0answers
248 views
0
votes
0answers
27 views

Mean encoding and covariate shift

I am working on a binary classification problem whose two main issues are categorical variables with many levels and, since the process, I am modeling is not yet stable, the proportion among the two ...
0
votes
0answers
15 views

How to encode factor predictors in prediction models

The response variable as well as all predictor variables in my dataset are factors. I want to build a model for predicting the response variable. As I understand I have to first encode my predictor ...
0
votes
0answers
7 views

Include or exclude original features after encoding

I have some categorical features and they are encoded by different types of encoding (one-hot, label, target, etc). My question is whether you usually include the original categorical features with ...
0
votes
0answers
32 views

One hot encoding of continuous floats using keras backend

I am trying to convert a vector of floats into a matrix similar to one hot encoding, but I want this to happen in non-discrete space to retain gradients. Therefore I am only able to use keras backend ...
0
votes
0answers
10 views

How to encode features that encode regular values as well as special categorical values

I was recently playing around with the FICO explainable machine learning challenge dataset. In the dataset, there are a bunch of numerical features which have values values typically in the 0-100 ...
0
votes
0answers
66 views

Using OrdinalEncoder on entire Dataframe and Avoiding Header transformation?

I want to encode the string in a dataframe which also has float values (which I don't want to change). The each feature column has several different unique strings. ...
0
votes
0answers
23 views

Is random forest a kind of spatial feature encoding?

From the book "Deep Learning and Convolutional Neural Networks for Medical Image Computing" On the other hand, CNN models have been proved to have much higher modeling capacity, compared to the ...
0
votes
1answer
134 views

String indices must be integers

I was trying to encode the string values of the feature 'ProductCategory' into integer values but I got this error. Kindly help. And I would also like to ask if label-encoding this feature would not ...
0
votes
0answers
154 views

Pre-processing data to make predictions on deployed Sklearn model

I am new to Machine Learning. I have trained a ML model on the Diamond Prices Dataset to predict the price of a diamond given it's features (carat, cut color, clarity, etc...) I have used pickle to ...
0
votes
1answer
54 views

How to deal with name strings in large data sets for ML?

My data set contains multiple columns with first name, last name, etc. I want to use a classifier model such as Isolation Forest later. Some word embedding techniques were used for longer text ...
-2
votes
1answer
2k views

How to use same encode label with same value used in training

I did save my model and using that model I want to predict the data. I am using Flask HTTP server for prediction endpoint. I have training data like this. I did save my model and using that model I ...
-2
votes
1answer
823 views

Encoding categorical data 2 different columns

Suppose I have two columns namely Goods, and Quality which are to be one encoded. Goods                   &...
-3
votes
1answer
79 views

One Hot Encoding (OHE) for commodity price changes

I have a dataset of the historical price of a given commodity. What I would like to do is add a new column to the dataset that is a value of -1, 0, or 1 for each row of the dataframe. The -1,0,1 would ...