Multi-task learning for Multi-label classification?

Question

I have a multi-label classification problem wherein each example can belong to one of the pre-defined classes (or can belong to none of them).

I was wondering if I can somehow apply multi-task learning (MTL) to this problem. Essentially, treat each class as a separate class and use a NN based model with common feature extractor layer, and on top of it class specific layer.

My doubt is that generally there is a correlation between labels in a multi-label setting. In the MTL model, after feature extraction each class is handled separately without sharing information. Does a MTL kind of model for multi-label makes sense, given this information?

Answer 1

MTL is supposed to enable your shared layer to generalise better.For eg. in a text classification problem your encoder would be trained on separate tasks: one could be multi class classification(original task) itself and let's take identifying the sentiment of the sentence(again a classification task; but different than the original task) as the other task.

For each of these tasks encoder will be same but heads(decoders) will be different.

Decoder/head of one task won't be affected by the other task.

MTL always makes sense. Your encoder/shared layers will be able to generalise better putting less load on your decoders which in turn will increase the performance.