0
$\begingroup$

I wrote and trained my own SRGAN: so I obtained a generator’s model that takes 32x32 images as input and gives their improved 128x128 version as output…

However, the end users of my Android app will send images of any size, 3800x2800, 53x12, etc.

How can I run my SRGAN on such images: Should I change the generator’s training to take images inputs with any dimensions into account (differing from the original SRGAN research paper)? Or can I change the shape dimensions of the input layer of the model on the fly?

Note: https://deepai.org/machine-learning-model/torch-srgan 8 - they actually did it! I don’t know how…

Cutting the input in 32x32 patches

Two problems exist with this approach:

Are we sure the SRGAN can actually super-res a patch? Indeed, it’s a CNN network, thus it’s trained to recognize patterns. If we cut the image into several 32x32 patches, maybe a lot of patterns would be cut and, thus, unrecognizable. So the SRGAN would not be able to super-res them. Also, it’d not be able to super-res them for other reasons, due to this cutting too.

Considering the 1.'s answer is YES…: How can we deal with images that can’t be cut into 32x32 patches exactly? Imagine the simplest case: an image which is 33x32. We’d cut it into a patch of 32x32 and another patch of 1x32. The first one could be super-res, but not the last one. Of course it is a problem with only 1px but in real examples, this problem would appear with more than 1px.

Manually extending the input image to 32x32

This is an alternative solution I thought of. Imagine the Android app’s user can only send images whose width is <= 32, and whose height is <= 32. Then the app add black rows and cols of pixels around the (<=32 ; <=32) input image in order to send to the SRGAN a 32x32 image.

But the SRGAN doesn’t seem to appreciate it. This solution doesn’t work obviously. Indeed, I’ve tried it. I’ve trained my SRGAN on a set of only 1 image and at each epoch, I’ve output a test’s result. The test was realized with the black-cols-and-rows version of this training image. Result: the SRGAN wasn’t able to recognize that the training image was inside these blacks cols and rows. It could not super-res it. Then I used the training image also for the test (after having deleted the black-rows-and-cols version): the SRGAN could actually super-res it: so my SRGAN doesn’t bug (i.e.: the problem doesn’t come from it).

Finally…

What should I do? Are you really sure cutting it into 32x32 would work? How to deal with the problem I’ve written about this approach? Is there any other solution? Do you have any idea of how DeepAI did it (the Website is given in my post)?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.