I understand that in Faster R-CNN, the image is fed into a pre-trained CNN (such as VG16). So say I have a 37x50x512 feature map. Firstly, I assume that each feature map (37x50x1) is fed into the RPN? or does each of the 512 feature maps get fed in one at a time or together?
Furthermore, I read around that anchors are created by "points of the feature map". So so I have the images below, and I choose a 3x3 sliding window, and I want to create 9 anchor candidates for each feature map pixel.
So the image above, the middle feature map pixel (the grey dash) is used and the 3x3 sliding window is the original size of the anchor box, so we do scaling 3 times [1:1,1:1.5,1:2] and aspect ratios to get the 9 anchors for that feature map pixel.
Also the red square is the original size of the box and the different colours represent the aspect and scaling performed.
So if we have a 37x50 feature map, we will have 37x50x9 anchors which equals 16650. From here we use the IOU to get the negative and positive boxes such as IOU > 0.7 = Positive or object and IOU < 0.1 = Negative or background, and the rest are neutral.
From here, I believe we feed each anchor box candidate (after finding a max best candidates) into a two 1x1 convolutional network to predict an object/background using classification and a bounding box estimate using regression.
Is this the way to understand the process? Also just quickly if I want to find the anchor boxes for the top left hand corner feature map pixel, does the sliding window go outside the original feature map? or is padding used to perform this? E.g. the below image is what im trying to explain