# Image classification if rotated version same

I asked this question on stackoverflow but was advised to come here.

I have some images to classify. I see that Convolutional neural network may be best for this, e.g. here. However, for my images, their rotated versions (to any degree) is effectively same image. What should I do so that this aspect is also taken into account.

One method I thought is to create many rotated versions (say, every 30 degree rotation) of each training image and add them to train set with same label. Is there any other way? Also, will it help if I get each image in a circular area?

Two example images are shown below:

• Are you familiar with spatial transformers? – Media Aug 30 '18 at 10:34
• Not really but I see a lot of links on searching the net, some of which I intend to read. It will be helpful if you can summarize it in an answer here. – rnso Aug 30 '18 at 12:18
• I've had a bad headache. Remind me tomorrow. I'll help more :) – Media Aug 31 '18 at 17:40

Just encode your images first and do the classification on the encoded data (Wavelet transformation). Parts of similar image regions will result in very similar encoded matrix. Further denoising the encoded matrix by zeroing the absolute values that are less or equal some small constant should reduce overfitting and improve your accuracy.

Here an example for a simple Haar-Wavelet transform encoder:

First we take the R values of the RGB-image and divide it into 8x8 matrices such as this one: $\begin{pmatrix}{97,95,99,94,97,88,90,95}\\{88,86,89,90,95,96,99,98}\\{95,99,94,91,88,85,90,94}\\{78,75,84,86,80,74,77,99}\\{71,75,78,70,71,84,91,90}\\{67,74,69,70,65,68,74,70}\\{59,55,64,68,70,84,86,90}\\{79,82,88,90,85,82,88,90}\end{pmatrix}$

Now we begin with the first row: $\begin{pmatrix}{97,95,99,94,97,88,90,95}\end{pmatrix}$

Grouping two adjacent values we obtain the 4 groups:

{97,95},{99,94},{97,88},{90,95}

Next step is to calculate the average of each group:

(96, 96.5, 92.5, 92.5)

And the difference of the second value of each group and the average:

95-96 = -1, 94-96.5 = -2.5, 88-92.5 = -4.5, and 95-92.5 = 2.5

Putting them together: (-1, -2.5, -4.5, 2.5)

Attaching them to the list of our average values we get:

(96, 96.5, 92.5, 92.5, -1, -2.5, -4.5, 2.5)

Now we repeat that process, but only using the first 4 values:

Grouping: {96, 96.5}, {92.5, 92.5}

Averages and differences: (96.25, 92.5, 0.25, 0)

One last time the same process but now only using the first 2 values:

Grouping: {96.25, 92.5}

Average and difference: (94.375, -1.875)

Putting it all together we obtain the new values of the first row, where we can see, that one value was already zeroed out:

(94.375, -1.875, 0.25, 0, -1, -2.5, -4.5, 2.5)

The next steps would be just applying the above process on the other 7 rows and then doing the same for every column of the resulting 8x8 matrix. Then doing the same on every other 8x8 part of our RGB-image's R-matrix and then doing the same for the G-matrix and B-matrix. The resulting RGB-matrices will be very sparse because of all the zeroed values. We got rid of all low entropy pixels what is left are the core features of the objects from the image. Using some simple similarity metrics will result in a high accuracy classifier.

The whole process can be very easy implemented using a simple recursive function, or by using linear algebra and matrix multiplications. This is a lossless compression, meaning the resulted matrix still have all needed information to decode it and getting the original image. However it is possible to apply some denoising routines to get more zeroes, but doing it, information will be lost.

• Excellent description. Thanks. It will help if you can clearly mention that this method will also lead to rotated images being identified to same label or class. – rnso Aug 30 '18 at 12:14
• Actually it is very intuitiv, since we are averaging rowwise as well as columnwise. It doesn't matter how you rotate the object, it will be normalized by the averaging process. – Eugen Aug 30 '18 at 12:24
• I made my own code but it is slow. Which python package can I use for this. I find pywt / pywavelet package: pywavelets.readthedocs.io/en/latest . But should I use horizontal, vertical or diagonal details? Or any will do? – rnso Aug 31 '18 at 1:46
• You can use matrices and some linear algebra to speed it up. If D is your 8x8 data matrix you can do the first part from above (horizontal encoding) by using matrix multiplication: $T_1 = \begin{pmatrix}{\frac{1}{2}, 0, 0, 0, \frac{1}{2}, 0, 0, 0}\\{\frac{1}{2},0,0,0,-\frac{1}{2},0,0,0}\\{0,\frac{1}{2}, 0, 0, 0, \frac{1}{2}, 0, 0}\\{0,\frac{1}{2}, 0, 0, 0, -\frac{1}{2}, 0, 0}\\{0,0,\frac{1}{2}, 0, 0, 0, \frac{1}{2}, 0}\\{0,0,\frac{1}{2}, 0, 0, 0, -\frac{1}{2}, 0}\\{0,0,0,\frac{1}{2}, 0, 0, 0, \frac{1}{2}}\\{0,0,0,\frac{1}{2}, 0, 0, 0,- \frac{1}{2}}\\\end{pmatrix}$ – Eugen Sep 13 '18 at 7:59
• Next step: $T_2 = \begin{pmatrix}{\frac{1}{2}, 0, \frac{1}{2},0,0, 0, 0, 0}\\{\frac{1}{2},0,-\frac{1}{2},0,0,0,0,0}\\{0,\frac{1}{2}, 0, \frac{1}{2}, 0,0,0, 0}\\{0,\frac{1}{2}, 0, -\frac{1}{2}, 0,0,0, 0}\\{0,0,0, 0, 1, 0, 0, 0}\\{0,0,0, 0, 0, 1, 0, 0}\\{0,0,0, 0, 0, 0,1, 0}\\{0,0,0, 0, 0, 0, 0,1}\\\end{pmatrix}$ – Eugen Sep 13 '18 at 8:09