Questions tagged [categorical-encoding]

The tag has no usage guidance.

28 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
79 views

What is the concept behind the categorical-encoding used in the CatBoost benchmark problems?

I'm working through CatBoost quality benchmark problems (here). I'm particularly intrigued by the methodology adopted to convert categorical features to numerical values as described in the ...
PPR's user avatar
  • 171
2 votes
0 answers
234 views

Does it make sense to use target encoding together with tree-based models?

I'm working on a regression problem with a few high-cardinality categorical features (Forecasting different items with a single model). Someone suggested to use target-encoding (mean/median of the ...
KJA's user avatar
  • 21
2 votes
1 answer
446 views

Cat2Vec implementation X = categorical and y = categorical

I am trying to convert categorical values (zipcodes) with Cat2Vec into a matrix which can be used as an input shape for categorical prediction of a target with binary values. After reading several ...
Snader's user avatar
  • 21
2 votes
1 answer
392 views

One-hot & interaction one-hot on multiple categorical

I was wondering if there is any value to creating combined features out of multiple categorical variables when the individual categorical variables are already one-hot encoded? Simple example: there ...
Artur Motruk's user avatar
2 votes
1 answer
81 views

What to do if a specific label of a category appears only a few times?

Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
user avatar
2 votes
0 answers
329 views

Multi-valued categorical features in LIME

I am working with the LIME implementation by Marco Ribeiro (https://github.com/marcotcr/lime). Specifically, I am utilizing the LimeTabularExplainer as I have a mixture of numerical and categorical ...
BVB's user avatar
  • 21
1 vote
0 answers
47 views

Intuition behind catboost encoding techniques

Can anyone please help me in understanding the effect of various bucketing techniques used in CatBoost Algorithm for categorical features? Like there is border, buckets, binarized target mean, counter ...
Mimansa Maheshwari's user avatar
1 vote
0 answers
226 views

Encode each comma separated value in Pandas

I have a dataset ...
spd's user avatar
  • 119
1 vote
0 answers
136 views

Handling date and time fields for classification task

I'm working on a classification task(The dataset is 400,000 rows and 30 columns) and one of my features was date-time. I've extracted the month, day of the week, and hour from the dataset (year is a ...
insomniac's user avatar
1 vote
1 answer
1k views

Should one-hot encoded categorical features needs to be scaled when used along with text feature while deriving semantic similarity?

My aim is to derive textual similarity using multiple features. Some of the features are textual for which I am using (Tfhub 2.0) Universal Sentence encoder. There are other categorical features which ...
Bruso's user avatar
  • 111
1 vote
0 answers
339 views

Implementing Scikit Learn's FeatureHasher for High Cardinality Categorical Data

Background: I am working on a binary classification of health insurance claims. The data I am working with has approximately 1 million rows and a mix of numeric features and categorical features (all ...
Jacob Niederer's user avatar
1 vote
1 answer
212 views

How to do target encoding when data has repeated rows?

How can I do encoding for a category when data has repeated rows? Can I do target encoding? Or Can I utilize another encoding? I want to figure out how to include a categorical variable in a model to ...
pierround's user avatar
0 votes
0 answers
7 views

Best way to encode a tag column for clustering

I have a dataset which tells me a tech support case used a particular tech document. Every case has been tagged with which product it pertains to. Similarly tech documents are tagged with certain key ...
haldar55's user avatar
0 votes
0 answers
24 views

Binary classification using xgboost

Why when adding new features in my ADS for a binary classification using XGBOOST my score and uplift has decreased ? What is the best way to treate categorical features or other features in order that ...
Warda_IDRIS's user avatar
0 votes
0 answers
24 views

Encode multiple label categorical variables with consideration of the frequency and standardization

I'm currently working on a dataset containing, among other features, multi-labeled categorical data per person for the last 3 years. I'm not sure how to handle this kind of categorical data where ...
codade's user avatar
  • 1
0 votes
0 answers
25 views

Multi-class classification model unable to return desired outcome

I have a scenario of multi-class dataset with around 10 distinct classes of target. There are 3 categorical features each with multiple labels. If we check the data, each unique combination of feature ...
soumalya saha's user avatar
0 votes
0 answers
14 views

How to encode data set with all categorical predictors

I have a data set with all categorical predictors. They are 14 in number. If I do one-hot encoding, I would be getting more than 35 new features, which I think is not right due to the curse of ...
emekadavid's user avatar
0 votes
0 answers
534 views

Binary classification works with softmax, but not sigmoid

I am doing a binary classification problem for seizure classification. I split the data into Training, Validation and Test with the following sizes and shapes dataset_X = ...
Mohammed Nafie's user avatar
0 votes
1 answer
26 views

How do I use one hot encoding with 240 nominal variables and each with equal occurrence?

The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
learner's user avatar
0 votes
0 answers
66 views

Encode and replace missing categorical values

Should replacing missing values for categorical variables be done after one-hot-encoding? (ex: I have a column named Car Type: Sedan, Hatchback, MPV, SUV) which has 5% missing values. Would there be ...
Kusisi Karem's user avatar
0 votes
0 answers
119 views

how to handle categorical data that has two or more columns with unique values?

...
Deep Kumar Prasad's user avatar
0 votes
1 answer
875 views

Turning multiple binary columns into categorical (with less columns) with Python Pandas

I want to turn these categories into values of categorical columns. The values in each category are the current binary columns present in the data frame. We have : A11, A12.. is a detail of A1 so if ...
Legna's user avatar
  • 21
0 votes
1 answer
221 views

NAN in keras neural network results

I am creating a neural network simple architecture. But I keep getting NAN in result, cant figure out why, below is my code. ...
Ayan Mitra's user avatar
0 votes
1 answer
205 views

Collapsing categorical data into more than 3 categories

I have a bunch of categorical, part of speech data that I want to collapse into fewer categories. np.where() won't do because I want to have 6 categories at the end: noun, verb, adjective, adverb, ...
macarom's user avatar
0 votes
1 answer
427 views

What are the options/best practices for encoding categorical features for multilabel classification?

I am working on a multilabel classification problem with both continuous and categorical features. For a single label problem, I might make use of a supervised encoder for my categorical features such ...
tensormoby's user avatar
0 votes
1 answer
255 views

What is the best practice to normalize/standardize imbalanced data for outlier detection or binary classification task?

I'm researching Anomaly/outlier/fraud detection, and I'm looking for the best practice to pre-process the synthetic data for imbalanced data. I have checked all methodology for normalizing/...
Mario's user avatar
  • 424
0 votes
0 answers
650 views

Tensorflow - I don't get the right shapes - `ValueError: Shapes (9, 1) and (8, 9) are incompatible`

I want to train a Sequential Neural Net (NN) with Tensorflow. ...
Dominik's user avatar
0 votes
2 answers
117 views

Handling encoding of a dataset which has more than total 2000 columns

Whenever we have a dataset to be pre processed, before feeding it to the model we convert the categorical values to numerical values for which we generally use LabelEncoding, One Hot encoding etc ...
Sahil's user avatar
  • 101