Loss function for an RNN used for binary classification

Question

I'm using an RNN consisting of GRU cells to compare two bounding box trajectories and determine whether they belong to the same agent or not. In other words, I am only interested in a single final probability score at the final time step.

What I'm unsure about is how to formulate the loss function in this case. I see two options:

1) force the network to output the correct label at every time step i.e. if I am providing a positive training sample whose output should be 1, then my loss function would be a vector of ones subtracted by the network's output at each time step

2) only check the output at the final time step and use only that in my loss function.

Intuitively, the second option makes more sense, but I'm sure there are other factors that come into play too.

Answer 1

The second option is the good one.

• You can select the last output that correspond to a not padded input and using it for your loss.

• Or you can directly express that explicitly : in keras set the flag return_sequence to False and your RNN layer will only give you the last output that correspond to a not padded input.

If I were you I would try to put a dense layer between the RNN layer and the last output. Don't forget to use a softmax activation function for the last layer in order to get probabilities.