I have a suggestion for you. Maybe not complete enough to be a complete solution but that is exactly what I've experienced. Suppose that you have writings in a paper and the paper is a part of a scene, or you have boys with nice mustaches which are beside a wide scene. In such cases which you have special information which are significant for classifying, the first one may help the classifier recognize book and written things and the second one can help the classifier find the gender of people in the scene, try to resize the images in a way that such significant things be recognizable to the human. If humans can understand them, you can hope that your classifier will be able too. Consider that you shouldn't resize the images to small pixels to avoid your net being with lots parameters. If you resize the images to too small ones, you may loose important information.