2
$\begingroup$

I have a Keras Xception based model for gesture recognition. The accuracy of the model is around 60-70% for classifying 7 different gestures. The training dataset consists of 320x240 and 640x480 pixel images. Currently, I'm leaving the input_shape parameter of the model equal to default value of the Xception model in Keras, which is (299, 299, 3). I assume under the hood the network is rescaling all inputs to 299x299 pixels, which probably isn't a good approach.

My questions are:

  1. Is the Xception model somehow optimized for the 299x299 image size? That is to say, should I aim to crop/pad the input to 299x299 pixels rather than change the model's configuration?
  2. All usage examples I've seen so far have input width = height. Is there a reason to prefer square images?
  3. If I don't use cropping/padding, I have two options for the input shape: rescale all images to 640x480 in a preprocessing step, or rescale all to 320x240. Is the 640x480 option likely to result in much better accuracy?
$\endgroup$
2
$\begingroup$

For your first question, yes, it is optimized for that size since the original paper for Xception used 299x299 size. But, you can use other sizes. You should resize your images to 299x299 that would be the best.

For your second question, the reason height = width because in the network, the convolutional filters which are used are square (3x3 filters). The reason for using square filter in computer vision is we assume that the features in image are symmetric most of the times (exception being text where the information is more on the vertical dimension than horizontal, 1x2 filters are used there).

For your third question, go for the smaller size, because if you rescale the smaller images to bigger ones, you add useless information into image since it is derived from the smaller image itself. Also, you create a model with more parameters.

$\endgroup$
3
  • $\begingroup$ Please accept the answer, if it has solved your problem. $\endgroup$ Feb 25 at 13:08
  • $\begingroup$ Thanks, I upvoted it. But I have a question about the second point. Surely the 3x3 filters can also be applied to e.g. 320x240 pixel images? The filter size is much much less than image size in this case. $\endgroup$
    – kfx
    Feb 25 at 13:39
  • $\begingroup$ Yes, they can be but you can run experiment to see which works for you. Empirically, squared images with squared filters are used. Also, our love for symmetry plays as a bias in this decision. And as I have mentioned in the point that you can use non-square filters if your data has nature like that. But, then, you will have to drop Xception and train from scratch. So, when in Rome, do what the Romans do. Squari-fy your image and use it. $\endgroup$ Feb 25 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.