0
$\begingroup$

I have two types of images, meaning two different sets of possible labels. How can I use two different loss functions based on which image I am currently training on with a batch size of 1? Thanks !

$\endgroup$
0
$\begingroup$

This is not possible, for 2 reasons:

  1. It doesn't make mathematical sense. The Gradient must explore one loss function that is optimized to distinguish between more than one class. Using a loss function to classify a single class is equivalent to not learning anything.
  2. In ML, the models you train are meant to generalize on data they have never seen before. Because of this, you can't use different loss functions because once you test its performance on test data nobody knows to what class they belong.
| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your answer ! Maybe I wasn't clear in my question, but both images have more than one class, and are of almost the same object. $\endgroup$ – gypaetebarbu Jun 24 '19 at 15:54
  • $\begingroup$ In that case, you can train one model for every class that each image can belong to. And for each Network you can choose the loss function you prefer. $\endgroup$ – Leevo Jun 25 '19 at 7:17
0
$\begingroup$

One option is multi-task learning. In this case, multi-task learning would allow a single model to learn more than one independent label for the same image. There is a separate loss function for each task (i.e., independent label). Those separate loss functions would be aggregated for a single loss function to train the overall model.

Training on a batch size of one will result in overfitting. The network will optimize for each image individually. Increasing batch will regularize, increasing generalization, across images.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.