2
$\begingroup$

According to Keras documentation, sample_weight can be used in order to give any sample in the training data a different importance in the loss.

I have googled around but have not found the answer to my question as follows:

  • Do sample weights take part in the derivatives? In other words, does Keras use these weights to train the model or they just give rise to a different loss value? i.e., the derivatives are "still" computed w.r.t. to the "unweighted" loss?

Because the loss function is not actually defined based on the sample weights, its rather passed in (as an argument) to the fit function in Keras.

$\endgroup$
  • $\begingroup$ These weights don't take part as derivatives because these are constants.; These just help in loss balancing and the backprop that follows is accordingly $\endgroup$ – Aditya Mar 23 at 16:05
1
$\begingroup$

sample_weights affect the total_loss, which is used to calculate the gradients.

Because the loss function is not actually defined based on the sample weights, its rather passed in (as an argument) to the fit function in Keras

I think that what you're missing is that the loss function actually returns a list of loss values (one per sample in the batch). This is precisely because during fit, it is multiplied by the sample_weight of that batch.

Please note that although "class based" losses support reduction (over the batch), the Loss base class implements the weighting during reduction.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.