8 votes
Accepted

How to handle categorical variables with Random Forest using Scikit Learn?

Is one-hot encoding an option? It seems like no, due to the high cardinality of your feature, it might result in the course of dimensionality problems if your sample size is small and also if you are ...
Multivac's user avatar
  • 2,949
7 votes

Target encoding with KFold cross-validation - how to transform test set?

I have some doubts about the way how I should then encode test set. As there is no single mapping deduced from train set I think we should use the whole train set to fit the encodings and then use it ...
Carlos Mougan's user avatar
5 votes

On gradient boosting and types of encodings

This is actually a feature of tree-based models in general, not just gradient boosting trees. Not exactly a reference, but this Medium article explains why ordinal encoding is often more efficient. On ...
zachdj's user avatar
  • 2,684
5 votes

How do I encode the categorical columns if there are more than 15 unique values?

While most answers here suggest to use various encoding schemes, I would like to propose a different approach: collapsing categories. The idea is that if there are two (or more) similar categories, ...
Yossi Levy's user avatar
5 votes

How do I encode the categorical columns if there are more than 15 unique values?

If you have high cardinality categorical data(+10 distinct values) you can do Target Encoding. One hot Encoding in high cardinality scenarios has the following drawbacks: The input data for the model ...
Carlos Mougan's user avatar
5 votes

Decision Tree only splits to the left

This is the typical behavior if you have only one one-hot-encoded feature. Explanation With a single one-hot-encoded feature, the feature-vector has the form of an $n$-dimensional vector where ...
Broele's user avatar
  • 1,352
4 votes

How do I encode the categorical columns if there are more than 15 unique values?

You are right that most of the algorithms can digest only numerical data, i.e. the categorical features need to be converted to the numerical ones before running the regression. Besides ...
aivanov's user avatar
  • 1,500
4 votes

Strategies to encode categorical variables with many categories

Generally, the logic of the categorical count transformation lies in the fact that features with similar frequencies tend to behave similarly. Have words in a corpus as an example, common words share ...
Pluviophile's user avatar
  • 3,768
4 votes

How do I encode the categorical columns if there are more than 15 unique values?

[edit] See also Carlos' answer, I think it's better than mine. You should use one hot encoding for the categorical features. Replacing categorical values with numerical ones would be a bad idea, ...
Erwan's user avatar
  • 25.2k
4 votes
Accepted

Choose-many categorical features: alternatives to one-hot encoding?

If your algorithm is based on gradient descent optimization, you can use embeddings, which are dense representation spaces for discrete elements. Embeddings are supported by most deep learning ...
noe's user avatar
  • 25.6k
4 votes

How to handle categorical variables with Random Forest using Scikit Learn?

One-hot encoding (OHE) is the standard method to represent a categorical feature. In my opinion 200 is not high dimensionality, it's very common to use OHE on text data with a much higher number of ...
Erwan's user avatar
  • 25.2k
3 votes
Accepted

Is this attribute numeric or categorical (ordinal)? Help!

They are not categorical as they have a meaningful ordering that you likely want to use. The first is usable as is as it is roughly fraction religious times 10. Yes it's ordinal but happens to be just ...
Sean Owen's user avatar
  • 6,595
2 votes

Categorical and non-categorical data in the same column

Would it make sense to split the column into two where one is the categorical and another column just for the non-categorical? Absolutely yes. Split it into more than one column, that's the way to ...
Leevo's user avatar
  • 6,165
2 votes

Categorical and non-categorical data in the same column

If you provide more information about the detail of why there is no data for some data points that would make it easier. That being said, I would split it into three columns as follows: col_1: it ...
nimar's user avatar
  • 748
2 votes
Accepted

sklearn serialize label encoder for multiple categorical columns

LabelEncoder is meant for the labels (target, dependent variable), not for the features. OrdinalEncoder can be used for ...
Ben Reiniger's user avatar
  • 11.6k
2 votes

Can we optimize regression problems that have categorical variables by encoding them if on the other hand we are inserting multicollinearity?

Multicollinearity can be a problem if you choose to optimize linear regression with ordinary least squares (OLS). Because the data matrix $X$ can have less than full rank, therefore the moment matrix $...
Brian Spiering's user avatar
2 votes

Choose-many categorical features: alternatives to one-hot encoding?

There are many ways to encode categorical features in the category encoders library you can find many of them. The one that seems more promising given your data is target encodinc
Carlos Mougan's user avatar
2 votes

why should i do target encoding within cv loop?

From the performance point of view, you can one-hot encode the whole dataset before the model fitting and you will get identical results. Why people do it in the CV loop? Ideological perspective In ...
kate-melnykova's user avatar
2 votes
Accepted

How to Present All Categories in All Samples

The first thing we must accept that the sampling is probably doing the right job. What I mean is that if only 10% is being sampled then some unique value which is less than 5 can be easily missed. ...
10xAI's user avatar
  • 5,574
2 votes

kNN for non-ordinal variables

K-nearest neighbors algorithm (k-NN) can work with categorical features using counts as a distance metric. The nearest neighbors would have the closest count frequency. Also, color is ordinal. Each ...
Brian Spiering's user avatar
2 votes

Dealing with Extra Categories in Test Set

You can reserve a special ordinal value to indicate "unknown/unseen during training." You would use this special value for any and all values of x that ...
stepthom's user avatar
  • 121
2 votes

Dealing with Extra Categories in Test Set

Regarding the case when you have 'bad' category present in the entire dataset, I would recommend using sklearn.model_selection.train_test_split function, with stratify option set to a corresponding ...
Arislan Makhmudov's user avatar
2 votes

Encoding features with big amount of classes

big amount of classes are called High-cardinality refers to columns with values that are very uncommon or unique. Dealing on High Cardinality depends upon the data/use case/model, The Following are ...
Pluviophile's user avatar
  • 3,768
2 votes

Encoding features with big amount of classes

Usually when we have Categorical variables, we do one hot encoding to convert to numerical data and use in model. If we have n classes we get n variables. Now, High Cardinality Variables are those ...
Ashwiniku918's user avatar
  • 1,954
2 votes

How do I get the mean values that are greater than .5 for my model?

So if I understand you correctly you want to "one-hot-encode" or dummy-encode your variable "specialty" so that it goes from an interval scaled variable to a binary variable where ...
Fnguyen's user avatar
  • 1,733
2 votes
Accepted

Encoding for Linear Regression

Comments: You should not group any data even if there are duplicates, because this would distort the distribution of the values (features and target). OneHotEncoder...
Erwan's user avatar
  • 25.2k
2 votes

N-ary decision tree with categorical features

It looks like your features are not really categorical, at least age: with categorical features the possible values are known at training, so normally you cannot ...
Erwan's user avatar
  • 25.2k
2 votes

Encode 10k features where each feature is having more than 500 categories

There is no single best technique for anything. You will have to try multiple techniques and see which one gives the best result. Also since your categorical variables are all high cardinal variables, ...
spectre's user avatar
  • 2,020
1 vote

One hot encoding of target variable containing classes 1 to 9 not including zero

I don't know it this is an optimal approach. The natural way to solve a SUDOKU with computer science is Linear Programming. I am curious if a CNN will solve the problem. Can you add +1 or -1 in the ...
Cuca's user avatar
  • 68
1 vote

How do I encode the categorical columns if there are more than 15 unique values?

There are a number good options for encoding categorical data with high cardinality. If using Python, the Category Encoders package has like a dozen options as of this writing. I wrote a guide to ...
jeffhale's user avatar
  • 400

Only top scored, non community-wiki answers of a minimum length are eligible