Return to Answer

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are trying to build. Namely the two categories of model we will be considering are:

1. Tree Based Models: Gradient Boosted Decision Trees and Random Forests.
2. Non-Tree Based Models: Linear, kNN or Neural Network based.

Let's consider when to apply OHE and when to apply Label Encoding while building tree based models.

We apply OHE when:

1. When the values that are close to each other in the label encoding correspond to target values that aren't close (non  - linear data).
2. When the categorical feature is not ordinal (dog,cat,mouse).

We apply Label encoding when:

1. The categorical feature is ordinal (Jr. kg, Sr. kg, Primary school, high school  ,etc).
2. When we can come up with a label encoder that assigns close labels to similar categories: This leads to less splits in the tress hence reducing the execution time.
3. When the number of categorical features in the dataset is huge: One-hot encoding a categorical feature with huge number of values can lead to (1) high memory consumption and (2) the case when non-categorical features are rarely used by model. You can deal with the 1st case if you employ sparse matrices. The 2nd case can occur if you build a tree using only a subset of features. For example, if you have 9 numeric features and 1 categorical with 100 unique values and you one-hot-encoded that categorical feature, you will get 109 features. If a tree is built with only a subset of features, initial 9 numeric features will rarely be used. In this case, you can increase the parameter controlling size of this subset. In xgboost it is called colsample_bytree, in sklearn's Random Forest max_features.

In case you want to continue with OHE, as @AN6U5 suggested, you might want to combine PCA with OHE.

Lets consider when to apply OHE and Label Encoding while building non tree based models.

To apply Label encoding, the dependance between feature and target must be linear in order for Label Encoding to be utilised effectively.

Similarly, in case the dependance is non-linear, you might want to use OHE for the same.

Note: Some of the explanation has been referenced from How to Win a Data Science Competition from Coursera.

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are trying to build. Namely the two categories of model we will be considering are:

1. Tree Based Models: Gradient Boosted Decision Trees and Random Forests.
2. Non-Tree Based Models: Linear, kNN or Neural Network based.

Let's consider when to apply OHE and when to apply Label Encoding while building tree based models.

We apply OHE when:

1. When the values that are close to each other in the label encoding correspond to target values that aren't close (non-linear data).
2. When the categorical feature is not ordinal (dog, cat, mouse).

We apply Label encoding when:

1. The categorical feature is ordinal (Jr. kg, Sr. kg, Primary school, high school, etc).
2. When we can come up with a label encoder that assigns close labels to similar categories: This leads to less splits in the trees hence reducing the execution time.
3. When the number of categorical features in the dataset is huge: One-hot encoding a categorical feature with huge number of values can lead to (1) high memory consumption and (2) the case when non-categorical features are rarely used by model. You can deal with the 1st case if you employ sparse matrices. The 2nd case can occur if you build a tree using only a subset of features. For example, if you have 9 numeric features and 1 categorical with 100 unique values and you one-hot-encoded that categorical feature, you will get 109 features. If a tree is built with only a subset of features, initial 9 numeric features will rarely be used. In this case, you can increase the parameter controlling size of this subset. In xgboost it is called colsample_bytree, in sklearn's Random Forest max_features.

In case you want to continue with OHE, as @AN6U5 suggested, you might want to combine PCA with OHE.

Let's consider when to apply OHE and Label Encoding while building non tree based models.

To apply Label encoding, the dependance between feature and target must be linear in order for Label Encoding to be utilised effectively.

Similarly, in case the dependance is non-linear, you might want to use OHE for the same.

Note: Some of the explanation has been referenced from How to Win a Data Science Competition from Coursera.

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are trying to build. Namely the two categories of model we will be considering are:

1. Tree Based Models: Gradient Boosted Decision Trees and Random Forests.
2. Non-Tree Based Models: Linear, kNN or Neural Network based.

Let's consider when to apply OHE and when to apply Label Encoding while building tree based models.

We apply OHE when:

1. When the values that are close to each other in the label encoding correspond to target values that aren't close (non - linear data).
2. When the categorical feature is not ordinal (dog,cat,mouse).

We apply Label encoding when:

1. The categorical feature is ordinal (Jr. kg, Sr. kg, Primary school, high school ,etc).
2. When we can come up with a label encoder that assigns close labels to similar categories: This leads to less spilts in the tress hence reducing the execution time.
3. When the number of categorical features in the dataset is huge: One-hot encoding a categorical feature with huge number of values can lead to (1) high memory consumption and (2) the case when non-categorical features are rarely used by model. You can deal with the 1st case if you employ sparse matrices. The 2nd case can occur if you build a tree using only a subset of features. For example, if you have 9 numeric features and 1 categorical with 100 unique values and you one-hot-encoded that categorical feature, you will get 109 features. If a tree is built with only a subset of features, initial 9 numeric features will rarely be used. In this case, you can increase the parameter controlling size of this subset. In xgboost it is called colsample_bytree, in sklearn's Random Forest max_features.

In case you want to continue with OHE, as @AN6U5 suggested, you might want to combine PCA with OHE.

Lets consider when to apply OHE and Label Encoding while building non tree based models.

To apply Label encoding, the dependance between feature and target must be linear in order for Label Encoding to be utilised effectively.

Similarly, in case the dependance is non-linear, you might want to use OHE for the same.

Note: Some of the explanation has been referenced from How to Win a Data Science Competition from Coursera.

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are trying to build. Namely the two categories of model we will be considering are:

1. Tree Based Models: Gradient Boosted Decision Trees and Random Forests.
2. Non-Tree Based Models: Linear, kNN or Neural Network based.

Let's consider when to apply OHE and when to apply Label Encoding while building tree based models.

We apply OHE when:

1. When the values that are close to each other in the label encoding correspond to target values that aren't close (non - linear data).
2. When the categorical feature is not ordinal (dog,cat,mouse).

We apply Label encoding when:

1. The categorical feature is ordinal (Jr. kg, Sr. kg, Primary school, high school ,etc).
2. When we can come up with a label encoder that assigns close labels to similar categories: This leads to less splits in the tress hence reducing the execution time.
3. When the number of categorical features in the dataset is huge: One-hot encoding a categorical feature with huge number of values can lead to (1) high memory consumption and (2) the case when non-categorical features are rarely used by model. You can deal with the 1st case if you employ sparse matrices. The 2nd case can occur if you build a tree using only a subset of features. For example, if you have 9 numeric features and 1 categorical with 100 unique values and you one-hot-encoded that categorical feature, you will get 109 features. If a tree is built with only a subset of features, initial 9 numeric features will rarely be used. In this case, you can increase the parameter controlling size of this subset. In xgboost it is called colsample_bytree, in sklearn's Random Forest max_features.

In case you want to continue with OHE, as @AN6U5 suggested, you might want to combine PCA with OHE.

Lets consider when to apply OHE and Label Encoding while building non tree based models.

To apply Label encoding, the dependance between feature and target must be linear in order for Label Encoding to be utilised effectively.

Similarly, in case the dependance is non-linear, you might want to use OHE for the same.

Note: Some of the explanation has been referenced from How to Win a Data Science Competition from Coursera.

While @AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are trying to build. Namely the two categories of model we will be considering are:

1. Tree Based Models: Gradient Boosted Decision Trees and Random Forests.
2. Non-Tree Based Models: Linear, kNN or Neural Network based.

Lets consider when to apply OHE and when to apply Label Encoding while building tree based models.

We apply OHE when:

1. When the values that are close to each other in the label encoding correspond to target values that aren't close (non - linear data).
2. When the categorical feature is not ordinal (dog,cat,mouse).

We apply Label encoding when:

1. The categorical feature is ordinal (Jr. kg, Sr. kg, Primary school, high school ,etc).
2. When we can come up with a label encoder that assigns close labels to similar categories: This leads to less spilts in the tress hence reducing the execution time.
3. When the number of categorical features in the dataset is huge: One-hot encoding a categorical feature with huge number of values can lead to (1) high memory consumption and (2) the case when non-categorical features are rarely used by model. You can deal with the 1st case if you employ sparse matrices. The 2nd case can occur if you build a tree using only a subset of features. For example, if you have 9 numeric features and 1 categorical with 100 unique values and you one-hot-encoded that categorical feature, you will get 109 features. If a tree is built with only a subset of features, initial 9 numeric features will rarely be used. In this case, you can increase the parameter controlling size of this subset. In xgboost it is called colsample_bytree, in sklearn's Random Forest max_features.

In case you want to continue with OHE, as @AN6U5 suggested, you might want to combine PCA with OHE.

Lets consider when to apply OHE and Label Encoding while building non tree based models.

To apply Label encoding, the dependance between feature and target must be linear in order for Label Encoding to be utilised effectively.

Similarly, in case the dependance is non-linear, you might want to use OHE for the same.

Note: Some of the explanation has been referenced from How to Win a Data Science Competition from Coursera. Incase my explanation is incomplete or incorrect, please help me by suggesting corrections.

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are trying to build. Namely the two categories of model we will be considering are:

1. Tree Based Models: Gradient Boosted Decision Trees and Random Forests.
2. Non-Tree Based Models: Linear, kNN or Neural Network based.

Let's consider when to apply OHE and when to apply Label Encoding while building tree based models.

We apply OHE when:

1. When the values that are close to each other in the label encoding correspond to target values that aren't close (non - linear data).
2. When the categorical feature is not ordinal (dog,cat,mouse).

We apply Label encoding when:

1. The categorical feature is ordinal (Jr. kg, Sr. kg, Primary school, high school ,etc).
2. When we can come up with a label encoder that assigns close labels to similar categories: This leads to less spilts in the tress hence reducing the execution time.
3. When the number of categorical features in the dataset is huge: One-hot encoding a categorical feature with huge number of values can lead to (1) high memory consumption and (2) the case when non-categorical features are rarely used by model. You can deal with the 1st case if you employ sparse matrices. The 2nd case can occur if you build a tree using only a subset of features. For example, if you have 9 numeric features and 1 categorical with 100 unique values and you one-hot-encoded that categorical feature, you will get 109 features. If a tree is built with only a subset of features, initial 9 numeric features will rarely be used. In this case, you can increase the parameter controlling size of this subset. In xgboost it is called colsample_bytree, in sklearn's Random Forest max_features.

In case you want to continue with OHE, as @AN6U5 suggested, you might want to combine PCA with OHE.

Lets consider when to apply OHE and Label Encoding while building non tree based models.

To apply Label encoding, the dependance between feature and target must be linear in order for Label Encoding to be utilised effectively.

Similarly, in case the dependance is non-linear, you might want to use OHE for the same.

Note: Some of the explanation has been referenced from How to Win a Data Science Competition from Coursera.