I'm looking for a way to create a loss function that looks like this: The function should then maximize for the reward. Is this possible to achieve in Keras? Any suggestions how this can be achieved are highly appreciated.

def special_loss_function(y_true, y_pred, reward_if_correct, punishment_if_false):
    loss = if binary classification is correct apply reward for that training item in accordance with the weight
           if binary classification is wrong, apply punishment for that training item in accordance with the weight
    return K.mean(loss, axis=-1)

The approach I've been looking at for my example is to pass in the weights along with y_true and then cut the tensor into two, separating out the weights and the y_true as seen below. Would an approach like this be possible at all or would this interfere with the normalization process etc?

def decompose_y_true(y_true_and_weights):
    y_true = y_true_and_weights[:,1]
    weights= y_true_and_weights[:,1:]
    return y_true, weights

def custom_loss(y_true_and_weights, y_pred):
    y_true, y_weights = decompose_y_true(y_true_and_weights)
     loss = # some loss operation
    return K.mean(loss, axis=-1)

Much more elegant would be if I could pass in my weights over the sample_weights parameter in the fit function, but it seems there are some limits what shape those weights can have, and also there's no way to retrieve them within the loss function as far as I can tell. Or is there any way to pass this into the loss function somehow so I can operate on them from there?

  • $\begingroup$ Could probably be achieved using closures programiz.com/python-programming/closure $\endgroup$ Commented Nov 22, 2017 at 22:49
  • $\begingroup$ I don’t think closures will help here as the problem is that y_true is a tensor. Whatever is passed into that function will have to be operated on as tensor. $\endgroup$
    – Nickpick
    Commented Nov 25, 2017 at 21:08

2 Answers 2


You can write a function that returns another function, as is done here on GitHub

def penalized_loss(noise):
    def loss(y_true, y_pred):
        return K.mean(K.square(y_pred - y_true) - K.square(y_true - noise), axis=-1)
    return loss

input1 = Input(batch_shape=(batch_size, timesteps, features))
lstm =  LSTM(features, stateful=True, return_sequences=True)(input1)
output1 = TimeDistributed(Dense(features, activation='sigmoid'))(lstm)
output2 = TimeDistributed(Dense(features, activation='sigmoid'))(lstm)
model = Model(input=[input1], output=[output1, output2])
penalized_loss(noise=output1)], optimizer='rmsprop')

In the wrapper function you can pass scalars or keras tensors like additional inputs. In your case you would add the weights in the top wrapper function and reference them in your inward function.

  • $\begingroup$ I see, that's helpful, but can I reference them if they are not a scalar but a tensor with rank 3, so that different weights are applied if in a cross entropy the result is correct or incorrect, and also different weights for each sample? $\endgroup$
    – Nickpick
    Commented Nov 27, 2017 at 16:44
  • $\begingroup$ Wha if I don't want to put in a constant as noise but a tensor, do I need to initialize the tensor as variable somehow? $\endgroup$
    – Nickpick
    Commented Nov 28, 2017 at 21:35

I think the best solution is:

add the weights to the second column of y_true and then:

def custom_loss(y_true, y_pred)
    weights = y_true[:,1]
    y_true = y_true [:,0]

That way it's sure to be assigned to the correct sample when they are shuffled.

Note that the metric functions will need to be customized as well by adding y_true = y_true[:,0] at the top.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.