Questions tagged [categorical-encoding]
The categorical-encoding tag has no usage guidance.
31
questions with no upvoted or accepted answers
3
votes
0
answers
73
views
What is the concept behind the categorical-encoding used in the CatBoost benchmark problems?
I'm working through CatBoost quality benchmark problems (here). I'm particularly intrigued by the methodology adopted to convert categorical features to numerical values as described in the ...
- 171
2
votes
1
answer
375
views
Cat2Vec implementation X = categorical and y = categorical
I am trying to convert categorical values (zipcodes) with Cat2Vec into a matrix which can be used as an input shape for categorical prediction of a target with binary values.
After reading several ...
- 21
2
votes
1
answer
78
views
What to do if a specific label of a category appears only a few times?
Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
user83221
2
votes
0
answers
316
views
Multi-valued categorical features in LIME
I am working with the LIME implementation by Marco Ribeiro (https://github.com/marcotcr/lime). Specifically, I am utilizing the LimeTabularExplainer as I have a mixture of numerical and categorical ...
- 21
1
vote
0
answers
34
views
Intuition behind catboost encoding techniques
Can anyone please help me in understanding the effect of various bucketing techniques used in CatBoost Algorithm for categorical features? Like there is border, buckets, binarized target mean, counter ...
1
vote
0
answers
142
views
Does it make sense to use target encoding together with tree-based models?
I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model).
Someone suggested to use target-encoding (mean/median of the ...
- 11
1
vote
0
answers
194
views
1
vote
0
answers
88
views
Handling date and time fields for classification task
I'm working on a classification task(The dataset is 400,000 rows and 30 columns) and one of my features was date-time. I've extracted the month, day of the week, and hour from the dataset (year is a ...
- 13
1
vote
1
answer
820
views
Should one-hot encoded categorical features needs to be scaled when used along with text feature while deriving semantic similarity?
My aim is to derive textual similarity using multiple features. Some of the features are textual for which I am using (Tfhub 2.0) Universal Sentence encoder. There are other categorical features which ...
- 111
1
vote
1
answer
277
views
One-hot & interaction one-hot on multiple categorical
I was wondering if there is any value to creating combined features out of multiple categorical variables when the individual categorical variables are already one-hot encoded?
Simple example: there ...
- 11
1
vote
0
answers
304
views
Implementing Scikit Learn's FeatureHasher for High Cardinality Categorical Data
Background: I am working on a binary classification of health insurance claims. The data I am working with has approximately 1 million rows and a mix of numeric features and categorical features (all ...
1
vote
1
answer
188
views
How to do target encoding when data has repeated rows?
How can I do encoding for a category when data has repeated rows? Can I do target encoding? Or Can I utilize another encoding?
I want to figure out how to include a categorical variable in a model to ...
- 61
0
votes
0
answers
25
views
0
votes
0
answers
11
views
Encoding of categorical variables to reduce the effect of erroneous labels
I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, ...
- 11
0
votes
0
answers
17
views
Multi-class classification model unable to return desired outcome
I have a scenario of multi-class dataset with around 10 distinct classes of target. There are 3 categorical features each with multiple labels. If we check the data, each unique combination of feature ...
0
votes
0
answers
14
views
How to encode data set with all categorical predictors
I have a data set with all categorical predictors. They are 14 in number. If I do one-hot encoding, I would be getting more than 35 new features, which I think is not right due to the curse of ...
0
votes
0
answers
302
views
Binary classification works with softmax, but not sigmoid
I am doing a binary classification problem for seizure classification. I split the data into Training, Validation and Test with the following sizes and shapes
dataset_X = ...
0
votes
0
answers
10
views
How to change label values in mask image
I have a mask image, where 0,10,20,30 are the labels assigned to the image. I want to change that value to 0,1,2,3 for multiple images.
0 - background
10- Building
20- Forest
30- Road
- 140
0
votes
1
answer
23
views
How do I use one hot encoding with 240 nominal variables and each with equal occurrence?
The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
- 1
0
votes
0
answers
25
views
OHE vs dummy variables
Could someone explain to me the difference between creating categorical “embeddings” with StringIndexer and OneHotEncoder, vs just creating dummy variables for each category? Aside from it being more ...
- 1
0
votes
0
answers
39
views
Irreversible Hash Encoding vs Reversible One-Hot Encoding
I understand that hash encoding for categorical features allows us to restrict or limit the dimensions of our dataset, which one-hot encoding does not.
I was wondering how we could compare these two ...
0
votes
0
answers
52
views
Encode and replace missing categorical values
Should replacing missing values for categorical variables be done after one-hot-encoding? (ex: I have a column named Car Type: Sedan, Hatchback, MPV, SUV) which has 5% missing values.
Would there be ...
- 161
0
votes
0
answers
79
views
0
votes
1
answer
643
views
Turning multiple binary columns into categorical (with less columns) with Python Pandas
I want to turn these categories into values of categorical columns. The values in each category are the current binary columns present in the data frame. We have : A11, A12.. is a detail of A1 so if ...
- 21
0
votes
1
answer
162
views
NAN in keras neural network results
I am creating a neural network simple architecture. But I keep getting NAN in result, cant figure out why, below is my code.
...
- 129
0
votes
1
answer
133
views
Collapsing categorical data into more than 3 categories
I have a bunch of categorical, part of speech data that I want to collapse into fewer categories. np.where() won't do because I want to have 6 categories at the end: noun, verb, adjective, adverb, ...
- 1
0
votes
1
answer
43
views
What's the minimum percentage of categories should be present in the categorical variable for to ignore the variable entirely
For example, if i have a feature "colour_codes" that has close to 5000 distinct color codes inside it. And the number of samples/rows is 10 million. Then should I ignore the feature "...
- 13
0
votes
1
answer
273
views
What are the options/best practices for encoding categorical features for multilabel classification?
I am working on a multilabel classification problem with both continuous and categorical features. For a single label problem, I might make use of a supervised encoder for my categorical features such ...
- 63
0
votes
1
answer
191
views
What is the best practice to normalize/standardize imbalanced data for outlier detection or binary classification task?
I'm researching Anomaly/outlier/fraud detection, and I'm looking for the best practice to pre-process the synthetic data for imbalanced data. I have checked all methodology for normalizing/...
- 335
0
votes
0
answers
614
views
Tensorflow - I don't get the right shapes - `ValueError: Shapes (9, 1) and (8, 9) are incompatible`
I want to train a Sequential Neural Net (NN) with Tensorflow.
...
- 1
0
votes
2
answers
73
views
Handling encoding of a dataset which has more than total 2000 columns
Whenever we have a dataset to be pre processed, before feeding it to the model we convert the categorical values to numerical values for which we generally use LabelEncoding, One Hot encoding etc ...
- 101