I am trying to understand the difference between the two label encoding techniques for output variable. I have read things but still can't get a clear picture as what makes them different. Also can we apply them on independent variables.
This is a project I am working on where I have to predict Roles
for each observation. There are around 15k unique roles and each output variable has 3-20 combination of roles. This is where the binarizer comes into play.
Now when I do this
m=MultiLabelBinarizer()
ytrain=m.fit_transform(ytrain)
yval=m.transform(yval)
this is a warning I get but I understand that these labels are not present in my training set also this would avoid leakage but at the same time wouldn't this affect my model's performance.
'Senior Android Engineer', 'Senior Asset Manager', 'Senior Billing Manager', 'Senior Buyer', 'Senior Finance Analyst', 'Senior Manager Technical', 'Senior Nodejs Developer', 'Senior Optometrist', 'Senior Procurement Manager', 'Senior Revenue Assurance', 'Senior Visual Designer', 'Service Delivery Consultant', 'Shop Assistant', 'Site Deployment', 'Software Lead Tester', 'Solar Design Engineer', 'Sous Chef', 'Specialist Support Worker', 'Storage And Backup Engineer', 'Subject Matter Expert Data Science', 'Support Accountant', 'Support Engineer Information Technology', 'Sustainability Coordinator', 'System Support Specialist', 'Systems Developer', 'Team Lead Business Developer', 'Team Leader IT', 'Technical Operations Manager', 'Telecommunications Engineer', 'Telemetry Technician', 'Training and Competence Manager', 'Tutor', 'VP Production', 'Vice President Internal Audit', 'Vice President Investor Relations', 'Visual FoxPro Developer', 'Web Marketing Specialist', 'Web Producer', 'Welding Inspector', 'Work Life'] will be ignored
warnings.warn('unknown class(es) {0} will be ignored'
So I would like to know if this is the correct way of using MultiLabelBinarizer
or there is some other way to handle.
Any links, modules or answers is appreciated. Thankyou!!
iterative_train_test_split
i am getting this warningLabel not 2120 is present in all training examples.
$\endgroup$