I have a multitask network taking one input and trying to achieve two tasks (with several shared layers, and then separate layers).

One task is multiclass classification using the CrossEntropy loss, the other is sequence recognition using the CTC loss.

I want to use a combination of the two losses as criterion, something like Loss = λCE + (1-λ)CTC. The thing is that my CE loss starts around 2 while the CTC loss is in the 400s.

Should I rescale the losses at each epoch with a Max(L₁)/L₁ factor, where Max(L₁) is the maximal loss at epoch 1 and L₁ is each “sub-loss” at epoch 1. That is we scale the loss so that at the first epoch they have the same magnitude and then we keep scaling using those factors.

Is there a better approach? How do I ensure that my two losses have the same influence on the backpropagation with respect to λ?

  • $\begingroup$ Hi @silverduck. Can you upvote my answer too, trying to build some reputation 😁 $\endgroup$ – ashutosh singh Apr 21 at 14:18

Check this.

Under the heading Multi-tasks losses they have mentioned how they average losses from two different tasks. They do a weighted average depending on their use case.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.