Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Suppose I have this custom loss:
def custom_loss(y_true, y_pred):
out = K.clip(y_pred, 1e-8, 1-1e-8)
log_lik = y_true*K.log(out)
return K.sum(-log_lik*advantages)
How does keras (with TF as backend) know how to differentiate in terms of the input specifically and ignore the 'advantages' simply as coefficient? Does it do it numerically? If so is that the same as with it's own loss functions or only with custom?
keras will differentiate automatically by looking at the graph of operations you use as part of your custom loss function. Since you are using only operations that keras "knows about" aka that exist as operations in TensorFlow keras will automatically a graph of operations to backpropagate against.
If you however have a custom loss function that uses an operation TF doesn't know, you'd have to write a custom operation so that TF knows how to backpropagate against it
