I am attempting to do a two-fold task. The input is an image and based on the input I want to pick another image from a set of images (classification task) and then use both the images to obtain an output tensor. Clearly, I can train both the models separately if I know the ground truth of which image I should pick from that set. But, I only have the output tensor ground truth.

The problem, as it appears to me, is that if we employ a classification layer, the gradients will not be differentiable anymore. How do I deal with this problem? Is there literature which uses this kind of architecture for any application?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.