I am trying to understand the difference between the two label encoding techniques for output variable. I have read things but still can't get a clear picture as what makes them different. Also can we apply them on independent variables.

This is a project I am working on where I have to predict Roles for each observation. There are around 15k unique roles and each output variable has 3-20 combination of roles. This is where the binarizer comes into play.

Now when I do this



this is a warning I get but I understand that these labels are not present in my training set also this would avoid leakage but at the same time wouldn't this affect my model's performance.

'Senior Android Engineer', 'Senior Asset Manager', 'Senior Billing Manager', 'Senior Buyer', 'Senior Finance Analyst', 'Senior Manager Technical', 'Senior Nodejs Developer', 'Senior Optometrist', 'Senior Procurement Manager', 'Senior Revenue Assurance', 'Senior Visual Designer', 'Service Delivery Consultant', 'Shop Assistant', 'Site Deployment', 'Software Lead Tester', 'Solar Design Engineer', 'Sous Chef', 'Specialist Support Worker', 'Storage And Backup Engineer', 'Subject Matter Expert Data Science', 'Support Accountant', 'Support Engineer Information Technology', 'Sustainability Coordinator', 'System Support Specialist', 'Systems Developer', 'Team Lead Business Developer', 'Team Leader IT', 'Technical Operations Manager', 'Telecommunications Engineer', 'Telemetry Technician', 'Training and Competence Manager', 'Tutor', 'VP Production', 'Vice President Internal Audit', 'Vice President Investor Relations', 'Visual FoxPro Developer', 'Web Marketing Specialist', 'Web Producer', 'Welding Inspector', 'Work Life'] will be ignored
  warnings.warn('unknown class(es) {0} will be ignored'

So I would like to know if this is the correct way of using MultiLabelBinarizer or there is some other way to handle.

Any links, modules or answers is appreciated. Thankyou!!

  • 1
    $\begingroup$ One issue is that there are labels in the validation dataset that are not in the training set. The model can not make a prediction for those labels. A way to fix that is multi-label data stratification scikit.ml/stratification.html $\endgroup$ Oct 1, 2021 at 15:34
  • $\begingroup$ @BrianSpiering even after using iterative_train_test_split i am getting this warning Label not 2120 is present in all training examples. $\endgroup$
    – 10sha25
    Oct 1, 2021 at 22:31

2 Answers 2


Both are within one-vs-all scheme when there is a classification task.

LabelBinarizer it turn every variable into binary within a matrix where that variable is indicated as a column. In other words, it will turn a list into a matrix, where the number of columns in the target matrix is exactly as many as unique value in the input set. If your input labels look like [1, 4, 5] the resulting matrix, is a 3 column matrix and each 1, 4, 5 are a column. then if your instances (observations) are either of 1,4,5, it is gonna be indicated (binary) whether that instance correspond to label 1 or 4 or 5.

you use LabelBinarizer to build regular classifier, for example to train a logistic regression and create the response variable you can use

from sklearn.preprocessing import LabelBinarizer
lb = LabelBinarizer()
lb.fit_transform(['yes', 'no', 'no', 'yes'])

the output is


or if your feature column is ['red', 'red', 'green', 'blue', 'blue']

array([[1, 0, 0],
       [1, 0, 0],
       [0, 1, 0],
       [0, 0, 1],
       [0, 0, 1]])

MultiLabelBinarizer - does the similar thing but when you have multiple lables. when do you have multiple labels ? for example when you are doing mu multiple label classification. Say, you are building a classifier to predict tags for Questions on StackoverFlow. Your data looks like this

   qId              Tag
0   1                       c#
1   2                     python
2   2                 machine_learning
3   2                     pandas
4   2                      nlp

but you have to convert it in a format where you can do machine learning (one row per observation)

qId c# python machine_learning pandas nlp
1 1 0 0 0 0
2 0 1 1 1 1

and you will use

import pandas as pd
from sklearn.preprocessing import MultiLabelBinarizer

question_tags = pd.read_csv("question_tags.csv")
mlb = MultiLabelBinarizer()

I hope this clear out the differences when it comes to the practice

UPDATE on your case

how do you parse your 15K unique role to get those 3 category or combination ? is it like, seniority, department, role ? if so, shouldn't you make it like

question_tags = [{'Senior', 'Android', 'Engineer'}, {'Senior', 'Asset', 'Manager'}, {'Senior', 'Billing', 'Manager'}] 

and then pass it to the

mlb = MultiLabelBinarizer()
res = pd.DataFrame(mlb.fit_transform(question_tags), columns=mlb.classes_)

and you will end up with

enter image description here

which shows all three are senior, number 2 and 3 are managers and so on ?


if you don't have it parse and basically just need to encode each of 15K unique label, you go with binary. For example you have four observation where two of them are senior android engieers.

question_tags = ['Senior Android Engineer','Senior Android Engineer', 'Senior Asset Manager',  'Senior Billing Manager'] 
lb = LabelBinarizer()
pd.DataFrame(lb.fit_transform(question_tags), columns = lb.classes_)

enter image description here

  • $\begingroup$ so should I fit mlb only on (training set and only transform testing set) or fit it on (training and testing set) both. @user702846 $\endgroup$
    – 10sha25
    Oct 1, 2021 at 6:28
  • $\begingroup$ both have attribute of fit and transform means you can used them both for training and test. As they are both carrying LABEL - it means one uses them for RESPONSE VARIABLE or the OUTPUT - however, you can also use them to encode your X matrix but it depends on the case and how you would like to go about it. $\endgroup$
    – user702846
    Oct 1, 2021 at 9:50
  • $\begingroup$ @10sha25 you can add your case as an example in the question and say which one should you use - so I can add that part into the answer as well if you like ... $\endgroup$
    – user702846
    Oct 1, 2021 at 9:50
  • $\begingroup$ I have edited my question. @user702846 $\endgroup$
    – 10sha25
    Oct 1, 2021 at 10:12
  • $\begingroup$ I can't do that since each observation may have all 'Senior Android Engineer', 'Senior Asset Manager', 'Senior Billing Manager', 'Senior Buyer', 'Senior Finance Analyst', 'Senior Manager Technical', 'Senior Nodejs or just 'Senior Nodejs. If I do the way you mentioned how can I retrieve colname Android Engineer as a whole term which predicting $\endgroup$
    – 10sha25
    Oct 1, 2021 at 14:40

Scikit-learn's LabelBinarizer converts input labels into binary labels, each example belongs to a single class or not.

Scikit-learn's MultiLabelBinarizer converts input labels into multilabel labels, each example can belong to multiple classes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.