How do I perform weighted loss in multiple outputs on a same model in Tensorflow? This means I am using a model that is intended to have 3 outputs. I did this because I would like the network to learn the relationships of the input variables. More specifically, this is multi-output regression.

The output model shape is like the following:

tf.Tensor: id=606, shape=(3, 3), dtype=float32, numpy=
array([[ 0.01483282,  2.9515972 , -0.07268244],
       [-0.8814691 ,  2.543654  ,  0.08576971],
       [-0.6933001 ,  1.3419302 , -0.25192362]], dtype=float32)

The model.summary() is below:

Layer (type)                 Output Shape              Param #   
simple_rnn (SimpleRNN)       (3, 32)                   1152      
dense (Dense)                (3, 3)                    99        
Total params: 1,251
Trainable params: 1,251
Non-trainable params: 0

The shape is (3,3) with 1st index being the batch size (the network is an RNN). I need to set weighted loss for each of the outputs. I tried the argument weighted_losses on tf.keras.models.Model but the error returned:

ValueError: When passing a list as loss_weights, it should have one entry per model output. The model has 1 outputs

What is the correct method to have weighted loss using a single model?


1 Answer 1


As I understand it, you want to have a model trained on multiple "tasks". If so, this is what it could look like:

input_data = <your input sequence input>
output_data = <array of size (N, 3)>

out1_weight, out2_weight, out3_weight = <your weights for each output to adjust loss contribution>

input = Input()
simple_rnn = SimpleRNN()(input)

# Define the three outputs
out1 = Dense(1)(simple_rnn)
out2 = Dense(1)(simple_rnn)
out3 = Dense(1)(simple_rnn)

model = Model(inputs=[input], outputs=[out1, out2, out3], loss_weights=[out1_weight, out2_weight, out3_weight])

model.compile(<whatever params>)
model.fit(input_data, [output_data[:, 0], output_data[:, 1], output_data[:, 2]])

  • $\begingroup$ As an improvement to the template, The Input layer has input_shape and batch_size, input = tf.keras.layers.Input(input_shape, batch_size=<integer>) For the creation of Model, the argument loss_weights is part of the model.compile() method. $\endgroup$
    – arjepak
    Jun 7, 2019 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.