There is a multi-task problem that I try to solve with a single neutral network. In general it works fine, but it seems like there is a room for improvement.

The final loss to optimize looks like this

$loss = (task_1loss + task_2loss + task_3loss)/3$

Of course, all losses are on different scales, and as result, the overall loss is dominated by the highest loss. I wonder if there is best practices how to normalize tasks losses or parametrize them.

Thank you.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.