YOLOv3 Predicting bounding boxes for grid containing multiple centers

Question

I recently started learning about YOLO and object detection, and I am kind of stuck on something. I was wondering if someone could explain to me what happens when a grid cell contains the centers of more than one bounding box. Does the grid cell still predict a single bounding box?

In YOLOv3 the image is divided into several grid cells of size $13\times13$, $26\times26$ and $52\times52$ Each of these grid cells are responsible for predicting a bounding box if the center of the ground truth bounding box is in the grid cell.

I know that each grid cell predicts $B$ boxes, where $B$ is the number of anchor boxes. The dimensions of each of these predictions is $4+1+n$, where $n$ is the number of classes. $4$ for $b_x,b_y,b_w,b_h$, and 1 for the objectness score. $b_x,b_y,b_w,b_h$ is defined below:

$$ b_x = \sigma(t_x)+c_x \\ b_y = \sigma(t_y)+c_y \\ b_h = e^{t_h}p_h \\ b_w = e^{t_w}p_w $$

$p_h$ and $p_w$ is the height and width of the anchor boxes. $t_x, t_y, t_h, t_w$ are predictions made from a single anchor box of a grid cell. My confusion is when a grid cell contains the centers of more than one bounding box, there are more than one values for $t_x, t_y, t_h, t_w$, but the description of the YOLOv3 training process or the above equations does not seem to take this into account.

Answer 1

It depends on the combination of thresholds of the objectness score and non-max suppresion (NMS):

Objectness score $C$

Apart from the location and dimensions predictions ($t_x, t_y, t_h, t_w$) for each box, an objectness score $C$ is also predicted for each of the boxes. It tells us how likely an object is inside of a certain predicted bounding box (BB).

But How much it needs to be to ensure an object is inside?

This has not a precise answer, instead we use a certain threshold. For example, we could set that a BB contains an object if and only if the objectness $C>0.5$ (threshold = $0.5$). This value depends on the implmentation and it can be asked to the user as in this repo

Non-max suppresion (NMS)

Now that we know that there is an objectness score threshold, we can rephrase the question in the following way:

What if two BBs of the same grid cell have an objectness score $>$ threshold? Would one of these two BBs be eliminated?

This depends on the threshold used in non-max suppression. If this concept is new to you, I recommend you this tutorial about it.

Broadly speaking it's used to discard bounding boxes that have a lower objectness score than the bounding box assigned to the same class that has the highest objectness score, and that have an overlap with it bigger than the threshold we are talking about now.

This means that higher thresholds used in NMS will lead to discarding a fewer number of boxes (we will be allowing a bigger overlapping between the boxes).

Conclusion

So, to sum up, In the situation of the question (more than one object per grid cell). Only boxes that have values of $C$ higher than its threshold will be considered. And out of these boxes, if there are some that are assigned to the same class then the box that have the highest $C$ will be considered and also the ones that don't get eliminated after applying non-max suppresion.