I am currently working on a binary classification task where the class is imbalanced.

I have the following categorical attributes with different levels:

time_slot: 8 levels
product_type: 3 levels
state: 40 levels
due_day: 6 levels (Mon - Sat)
lead_time: numerical in days (0-100)

Now, I am planning to use three algorithms to start with:

Logistic Regression, Decision Tree and Random Forest

I am confused as to what sort of encoding strategy is best when it comes to categorical variables?

LabelEncoder, OneHot, BinaryEncoding?

Also, I am thinking of creating bins for lead_time

any pointers/tips will be useful.


The best option for encoding - OneHot, because if you use Label encoding you indicate that categorical values are comparable(for example label 1 < label 2), which most probably it's not true. One hot encoding create columns for each specific value in the column, moreover, these columns are linearly independent, so you don't create fake order between categorical values. Unfortunately, you got a lot of columns so learning of algorithms could be greedy for time and resource.

More details you will find there

| improve this answer | |
  • $\begingroup$ Thanks. If I use OHE, the columns will increase but what sort of algorithm will work on such sparse data? $\endgroup$ – chintan s May 20 at 14:01
  • $\begingroup$ Not a simple answer. If you use OHE, you could learn the algorithm by chunk of data frame. Another option's custom encoding, but it's the most robust way. Also you could try other options of encoding towardsdatascience.com/… $\endgroup$ – fuwiak May 20 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.