I have a multi-headed NN where one head performs multi-label classification and the other a regression task on a set of images. The classification head outputs a one-hot vector where each value in the vector represents a probability for the given class being present in the image. The regression head should provide a prediction of the percentage of the image that is covered by each class class.

So a dummy example of the target classes and network predictions would look like:

classes = ['tree', 'car', 'ground', 'sky']
class_prob_preds = np.array([0.03, -0.17, 0.58, 0.73])
area_preds = np.array([0.05, 0.01, 0.42, 0.50]) # regression values don't need to add to 1
class_preds = np.where(class_prob_preds > 0.5) # gives the index of the predicted classes

My question is, how should I pass to the loss function for the regression part? Should I feed the entire vector of area predictions to the regression loss function, or should I use the predicted classes of the classification head to select only the values in the regression vector that correspond to the predicted classes and pass these to the loss function?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.