I have a multi-label classification problem wherein each example can belong to one of the pre-defined classes (or can belong to none of them).

I was wondering if I can somehow apply multi-task learning (MTL) to this problem. Essentially, treat each class as a separate class and use a NN based model with common feature extractor layer, and on top of it class specific layer.

My doubt is that generally there is a correlation between labels in a multi-label setting. In the MTL model, after feature extraction each class is handled separately without sharing information. Does a MTL kind of model for multi-label makes sense, given this information?

  • $\begingroup$ If your examples can only belong to at most 1 class, why do you even need multi-task here? A regular softmax would do - no? $\endgroup$ – Maverick Meerkat Nov 8 '19 at 13:17

MTL is supposed to enable your shared layer to generalise better.For eg. in a text classification problem your encoder would be trained on separate tasks: one could be multi class classification(original task) itself and let's take identifying the sentiment of the sentence(again a classification task; but different than the original task) as the other task.

For each of these tasks encoder will be same but heads(decoders) will be different.

Decoder/head of one task won't be affected by the other task.

MTL always makes sense. Your encoder/shared layers will be able to generalise better putting less load on your decoders which in turn will increase the performance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.