I'm using an RNN consisting of GRU cells to compare two bounding box trajectories and determine whether they belong to the same agent or not. In other words, I am only interested in a single final probability score at the final time step.

What I'm unsure about is how to formulate the loss function in this case. I see two options:

1) force the network to output the correct label at every time step i.e. if I am providing a positive training sample whose output should be 1, then my loss function would be a vector of ones subtracted by the network's output at each time step

2) only check the output at the final time step and use only that in my loss function.

Intuitively, the second option makes more sense, but I'm sure there are other factors that come into play too.


1 Answer 1


The second option is the good one.

  • You can select the last output that correspond to a not padded input and using it for your loss.

  • Or you can directly express that explicitly : in keras set the flag return_sequence to False and your RNN layer will only give you the last output that correspond to a not padded input.

If I were you I would try to put a dense layer between the RNN layer and the last output. Don't forget to use a softmax activation function for the last layer in order to get probabilities.

  • $\begingroup$ Unfortunately that wouldn't make sense in my case since my inputs of variable time lengths (I'm using TensorFlow's dynamic_rnn API). There are zeros padded to the end of each sequence to make the overall input batch a square matrix. $\endgroup$
    – Ali250
    Jul 12, 2018 at 8:12
  • $\begingroup$ I have updated the answer. Of course you still have to pad. You have to select the last "right" output. $\endgroup$
    – Adrien D
    Jul 12, 2018 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.