I'm trying to implement a custom ROI pooling layer in Keras. According to the Fast-RCNN publication, ROI pooling is done this way:

RoI max pooling works by dividing the $h \times w$ RoI window into an $H \times W$ grid of sub-windows of approximate size $h/H \times w/W$ and then max-pooling the values in each sub-window into the corresponding output grid cell. Pooling is applied independently to each feature map channel, as in standard max pooling. The RoI layer is simply the special-case of the spatial pyramid pooling layer used in SPPnets [11] in which there is only one pyramid level. We use the pooling sub-window calculation given in [11].

(where $H$ and $W$ are height and width of pool respectively, for the sake of simplicity, let $H=W=1$).

A paragraph above is quite understandable, but it contains implicit information on how exactly the pooling is applied to each feature map channel. This is better explained by the figure below:

enter image description here

Based on figure above, we can see that pooling to each feature map is applied by projecting a region of interest on each bounding box. Here's another illustration for projection in SPPNet:

enter image description here


How exactly is this projection performed? what if coordinates of region of interest is larger than the shape of feature map?

My solution:

I found out from the possibly original ROI pooling layer written in the Caffe, that they use something called "spatial scale". I believe the formula for spatial scale is $\frac{1}{N}$ where $N$ is the sum of all the strides of convolutional layers that image was processed by (in this case let image be a cowboy picture in the figure above).

This makes perfect sense I believe, spatial scale might be a subsampling ratio, or it should be at least associated with subsampling ratio.

Hence let's say we have a bounding box with position [x_start, y_start, x_end, y_end] and a single convolutional feature map of size $164x164$. We can then simply select a region [x_start * spatial_scale, y_start * spatial_scale, x_end * spatial_scale, y_end * spatial_scale] and then maxpool that region by diving it into $H \times W = 1 \times 1$ sub-windows.

Brief answer to my question:

By multiplying the coordinates by a spatial scale, we can make sure that it fits the feature map just like the original bounding box would fit the original image.

Is my answer correct? Is this correct way to do ROI maxpooling?

Thank you!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.