# Loss function for model with two-outputs

I have created a model for this Kaggle competition that outputs a classification of the level of the disease (from 0 to 4) from an image of the retina.

I now want to blend the predictions for both eyes from predictions for the individual eyes. Since the cause of the disease is organical, a lot of information from an eye can be extracted from the status of the other eye.

To blend the information about the level of the disease in both eyes I have created multiple predictions for each eye by feeding augmented versions of the image to the model and defined a model that generates a definitive prediction for both eyes from predictions for each eye.

To do so, the model takes as input a vector $$( x_1, x_2, \dots, x_9)$$ where $$( x_1 \dots x_5 )$$ are the predictions for the left eye, and $$( x_6 \dots x_{10} )$$ are the predictions for the right eye and it should output $$(y_1, y_2)$$, the final prediction for the left and right eye.

Which loss function should I use? I thought about using the sum of the cross-entropy loss on the left and right eye, is that reasonable?

• What is your goal? Evaluating how well each eye works? Are you doing multiple predictions and then combining them somehow to hopefully get a better prediction, but the end goal is still to predict the one thing? Feb 3 at 15:22
• @user2974951 Yes, the objective is to combine predictions to create a better prediction. Since the root cause of the disease is diabetes, usually both eyes have a similar level of the disease but it may not evident in both (because in that moment, there is an hemorraghe only in one eye, for example). The end goal is to predict the disease in an eye by eye basis.
– Zan
Feb 3 at 15:35
• In that case, similar to stacking, you would combine the predictions in some way, usually by simple average, and then pass it your loss function as usual. Feb 3 at 15:43
• @user2974951 I am not convinced about that. If the right prediction was (2,2) and the network outputs (0, 4) then the loss would be evaluated to 0, wouldn't it? But it has misclassified quite heavily both eyes
– Zan
Feb 3 at 15:50
• Ah, but now you are talking about evaluating each eye individually, which in this case were very wrong. But together, their average, was spot on - that is 2. And that is what you care about, is it not? Or do you indeed care about each eye's prediction? Feb 3 at 15:52