0
$\begingroup$

I have a multi-headed NN where one head performs multi-label classification and the other a regression task on a set of images. The classification head outputs a one-hot vector where each value in the vector represents a probability for the given class being present in the image. The regression head should provide a prediction of the percentage of the image that is covered by each class class.

So a dummy example of the target classes and network predictions would look like:

classes = ['tree', 'car', 'ground', 'sky']
class_prob_preds = np.array([0.03, -0.17, 0.58, 0.73])
area_preds = np.array([0.05, 0.01, 0.42, 0.50]) # regression values don't need to add to 1
class_preds = np.where(class_prob_preds > 0.5) # gives the index of the predicted classes

My question is, how should I pass to the loss function for the regression part? Should I feed the entire vector of area predictions to the regression loss function, or should I use the predicted classes of the classification head to select only the values in the regression vector that correspond to the predicted classes and pass these to the loss function?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.