There are some interesting literature about RPNs (Region Proposal Network). The most concise and helpful documentation that I found so far is the following: https://www.quora.com/How-does-the-region-proposal-network-RPN-in-Faster-R-CNN-work?share=1.
But there is something that I still don't understand through my various lectures. RPNs are designed to propose several candidate regions. From which, a selection will be done to know which candidates fits our needs.
But, RPNs and neural network in general are deterministic. Thus, once trained, they will always produce the same output for a given input; there is no way to query new candidates given the same input image. As far as I understood, RPNs are trained to produce a fix number of proposal, for each new image. But how the training work then? If the RPN has to produce 300 candidates, what should be the labeled data that we use for the training, knowing that a training image probably won't have more than 5 golden truth bounding boxes?
And then, knowing that the bounding box sizes are not consistent among candidates, how does the CNN behind operates with the different size of the input?