# How to encode a job description for machine learning

I'm working on a sample project and one of the features is the job description of a person (categorical, for example: blue-collar, retired, unknown, unemployed, student, etc.). Since in the future more job descriptions could be used, I don't think one hot encoding it's the best approach. How would you encode it without using one hot encoding??

• What's the goal of the project? If it's a sample project and you already have a dataset in hand, why worry about additional values in the future? Mar 23 '20 at 16:49

I would first start by using something like scikit-learn LabelEncoder. Examples are in the documentation.

EDITED : With this option, you convert every string to a integer starting from 1.

You can also think about trying to convert the strings into hash values.

• LabelEncoder outputs values between 1 and k, not 0 and 1. The problem with this is that by giving some class a greater value than other class, the model might be more biased towards the bigger value, which is what I'm trying to avoid. Apr 29 '19 at 13:29
• The problem with LabelEncoder pointed by @shulito is true. Apart from that, an algorithm could learn that the sum of two categories are the same as a third category, which would make absolutely no sense. I always take LabelEncoding with great care! It looks like a simple and efficient approach, but it may be dangerous! Oct 23 '19 at 22:38
• Yes my bad, after a few months, I don't know why I wrote 0 to 1. I will edit the answer. You are right, that you might have a biased model but I did have nice surprises in the past trying it anyway. Oct 25 '19 at 13:46

1) Try with LabelEncoding(fast way, but not best way)

#Assuming that you have data in Pandas dataframe

def encode_to_num_df(df):
from sklearn.preprocessing import LabelEncoder
df = df.apply(LabelEncoder().fit_transform)
return df


2) Custom encoding each label - best way, but the hardest. Detailed explanation there. Thanks to @Djib2011

my_mapping = {'bad': 1, 'worse': 2, 'worst': 3}

df['feature'] = df['feature'].map(my_mapping)


Source: