What is the difference between LabelBinarizer and MultiLabelBinarizer?

Question

I am trying to understand the difference between the two label encoding techniques for output variable. I have read things but still can't get a clear picture as what makes them different. Also can we apply them on independent variables.

This is a project I am working on where I have to predict Roles for each observation. There are around 15k unique roles and each output variable has 3-20 combination of roles. This is where the binarizer comes into play.

Now when I do this

m=MultiLabelBinarizer()

ytrain=m.fit_transform(ytrain)
yval=m.transform(yval)

this is a warning I get but I understand that these labels are not present in my training set also this would avoid leakage but at the same time wouldn't this affect my model's performance.

'Senior Android Engineer', 'Senior Asset Manager', 'Senior Billing Manager', 'Senior Buyer', 'Senior Finance Analyst', 'Senior Manager Technical', 'Senior Nodejs Developer', 'Senior Optometrist', 'Senior Procurement Manager', 'Senior Revenue Assurance', 'Senior Visual Designer', 'Service Delivery Consultant', 'Shop Assistant', 'Site Deployment', 'Software Lead Tester', 'Solar Design Engineer', 'Sous Chef', 'Specialist Support Worker', 'Storage And Backup Engineer', 'Subject Matter Expert Data Science', 'Support Accountant', 'Support Engineer Information Technology', 'Sustainability Coordinator', 'System Support Specialist', 'Systems Developer', 'Team Lead Business Developer', 'Team Leader IT', 'Technical Operations Manager', 'Telecommunications Engineer', 'Telemetry Technician', 'Training and Competence Manager', 'Tutor', 'VP Production', 'Vice President Internal Audit', 'Vice President Investor Relations', 'Visual FoxPro Developer', 'Web Marketing Specialist', 'Web Producer', 'Welding Inspector', 'Work Life'] will be ignored
  warnings.warn('unknown class(es) {0} will be ignored'

So I would like to know if this is the correct way of using MultiLabelBinarizer or there is some other way to handle.

Any links, modules or answers is appreciated. Thankyou!!

Answer 1

Both are within one-vs-all scheme when there is a classification task.

LabelBinarizer it turn every variable into binary within a matrix where that variable is indicated as a column. In other words, it will turn a list into a matrix, where the number of columns in the target matrix is exactly as many as unique value in the input set. If your input labels look like [1, 4, 5] the resulting matrix, is a 3 column matrix and each 1, 4, 5 are a column. then if your instances (observations) are either of 1,4,5, it is gonna be indicated (binary) whether that instance correspond to label 1 or 4 or 5.

you use LabelBinarizer to build regular classifier, for example to train a logistic regression and create the response variable you can use

from sklearn.preprocessing import LabelBinarizer
lb = LabelBinarizer()
lb.fit_transform(['yes', 'no', 'no', 'yes'])

the output is

array([[1],
       [0],
       [0],
       [1]])

or if your feature column is ['red', 'red', 'green', 'blue', 'blue']

array([[1, 0, 0],
       [1, 0, 0],
       [0, 1, 0],
       [0, 0, 1],
       [0, 0, 1]])

MultiLabelBinarizer - does the similar thing but when you have multiple lables. when do you have multiple labels ? for example when you are doing mu multiple label classification. Say, you are building a classifier to predict tags for Questions on StackoverFlow. Your data looks like this

   qId              Tag
0   1                       c#
1   2                     python
2   2                 machine_learning
3   2                     pandas
4   2                      nlp

but you have to convert it in a format where you can do machine learning (one row per observation)

qId	c#	python	machine_learning	pandas	nlp
1	1	0	0	0	0
2	0	1	1	1	1

and you will use

import pandas as pd
from sklearn.preprocessing import MultiLabelBinarizer

question_tags = pd.read_csv("question_tags.csv")
print(question_tags.head())
mlb = MultiLabelBinarizer()
print(mlb.fit_transform(question_tags))

I hope this clear out the differences when it comes to the practice

UPDATE on your case

how do you parse your 15K unique role to get those 3 category or combination ? is it like, seniority, department, role ? if so, shouldn't you make it like

question_tags = [{'Senior', 'Android', 'Engineer'}, {'Senior', 'Asset', 'Manager'}, {'Senior', 'Billing', 'Manager'}] 

and then pass it to the

mlb = MultiLabelBinarizer()
res = pd.DataFrame(mlb.fit_transform(question_tags), columns=mlb.classes_)

and you will end up with

which shows all three are senior, number 2 and 3 are managers and so on ?

UPDATE 2

if you don't have it parse and basically just need to encode each of 15K unique label, you go with binary. For example you have four observation where two of them are senior android engieers.

question_tags = ['Senior Android Engineer','Senior Android Engineer', 'Senior Asset Manager',  'Senior Billing Manager'] 
lb = LabelBinarizer()
pd.DataFrame(lb.fit_transform(question_tags), columns = lb.classes_)

Answer 2

Scikit-learn's LabelBinarizer converts input labels into binary labels, each example belongs to a single class or not.

Scikit-learn's MultiLabelBinarizer converts input labels into multilabel labels, each example can belong to multiple classes.