Which is better: multi-output model or separate models for similar tasks?

Question

I am working on two problems:

1. classification of images into high-level classes (e.g. shoe, dress, jacket etc.)
2. classification of the attributes of the same images on a lower level (e.g. shoe style, color of the dress etc.), assuming that the high level class is known

Currently, I have designed an architecture for the 2nd problem as a multi-class multi-output network with ResNet50 as the backbone. Now I am dealing with the 1st problem and I have two paths to follow:

1. consider both problems as different tasks and train separate models for them
2. share the backbone between 1st and 2nd problems and train only one multi-output model

Which option is better? Which one should work better? Are there any good practices for combining similar tasks?

Answer 1

If both the tasks consolidated classes set is small,using the multi-output network will be useful. But if your consolidated classes set is very large then I would suggest you go with different networks.

Personally, I would prefer to train individual models as it is hard to optimize multi-output networks.