1
$\begingroup$

I am attempting to do a two-fold task. The input is an image and based on the input I want to pick another image from a set of images (classification task) and then use both the images to obtain an output tensor. Clearly, I can train both the models separately if I know the ground truth of which image I should pick from that set. But, I only have the output tensor ground truth.

The problem, as it appears to me, is that if we employ a classification layer, the gradients will not be differentiable anymore. How do I deal with this problem? Is there literature which uses this kind of architecture for any application?

TIA

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.