Let's say I'm trying to train a neural network that predicts a single output [0.0, 1.0] value that correlates to photo realism which I can use either in a classification setting or for ranking. I have two datasets to work with.

Dataset 1: given an image, does it appear to be real {yes, no}

Dataset 2: given images A and B, which appears more realistic {A appears more realistic, B appears more realistic, they appear equally realistic}

I'm trying to figure out how to combine these datasets to train a model. I assumed I would just include examples from both in my training batches, compute two losses, one for the first dataset and one for the second, then just weight the losses 1 or 0 according to which dataset the example came from.

For Dataset 1 it seems a relatively straight forward Cross Entropy loss on the output would suffice.

What loss function would I apply to the output for Dataset 2? And do I need to add anything to make sure these two losses behave well together / one of them doesn't dominate the training? Everything I've read about ranking losses seem to involve measuring distances in an embedding space and pushing those further apart / closer together, which doesn't immediately sound applicable.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.