Is it possible to stack two networks on top of each other that operate on different resolutions of input data?

So here's my usecase: like Google, I want to recognize text in images. Unlike Google, I have access to very limited computational resources. My solution for this is that instead of using a sequence-to-sequence network on the whole scene image, I first run object detection using YOLO and then pass the crop to a sequence-to-sequence model, Attention-OCR.

To further reduce processing time, I run the object detection on low resolution, and crop the detection result in the higher resolution input image, so I still have access to the high resolution input when using sequence recognition on my crop. To detect that there is text, the low resolution is adequate, but to read the text, the network needs the higher resolution input.

This all works fine, but I suspect I would get better performance if I could train the whole system end-to-end, so the crop gets directly optimized to result in the best reading of the text. I could stack the sequence-to-sequence model on top of the object detection model to do this, but then the text reading operates on the same low resolution input that is used for the text detection.

Does anyone have an idea how to solve this, or can anyone point me to research that is related to this?