# How to project a bounding box on feature map?

I'm trying to implement a custom ROI pooling layer in Keras. According to the Fast-RCNN publication, ROI pooling is done this way:

RoI max pooling works by dividing the $$h \times w$$ RoI window into an $$H \times W$$ grid of sub-windows of approximate size $$h/H \times w/W$$ and then max-pooling the values in each sub-window into the corresponding output grid cell. Pooling is applied independently to each feature map channel, as in standard max pooling. The RoI layer is simply the special-case of the spatial pyramid pooling layer used in SPPnets [11] in which there is only one pyramid level. We use the pooling sub-window calculation given in [11].

(where $$H$$ and $$W$$ are height and width of pool respectively, for the sake of simplicity, let $$H=W=1$$).

A paragraph above is quite understandable, but it contains implicit information on how exactly the pooling is applied to each feature map channel. This is better explained by the figure below:

Based on figure above, we can see that pooling to each feature map is applied by projecting a region of interest on each bounding box. Here's another illustration for projection in SPPNet:

# Question:

How exactly is this projection performed? what if coordinates of region of interest is larger than the shape of feature map?

# My solution:

I found out from the possibly original ROI pooling layer written in the Caffe, that they use something called "spatial scale". I believe the formula for spatial scale is $$\frac{1}{N}$$ where $$N$$ is the sum of all the strides of convolutional layers that image was processed by (in this case let image be a cowboy picture in the figure above).

This makes perfect sense I believe, spatial scale might be a subsampling ratio, or it should be at least associated with subsampling ratio.

Hence let's say we have a bounding box with position [x_start, y_start, x_end, y_end] and a single convolutional feature map of size $$164x164$$. We can then simply select a region [x_start * spatial_scale, y_start * spatial_scale, x_end * spatial_scale, y_end * spatial_scale] and then maxpool that region by diving it into $$H \times W = 1 \times 1$$ sub-windows.