I came across this question from the 3rd chapter of the book Neural Networks and Deep Learning by Michael Nielsen, this is a question given in his exercise.
One way of expanding the MNIST training data is to use small rotations of training images. What's a problem that might occur if we allow arbitrarily large rotations of training images?
I would happy if someone explain why large rotations would be problematic.
The problem is that numbers are not invariant to rotations.
For example, see what happens when you rotate a 4 in steps of 90 degrees:
So unless your task includes recognizing numbers which are written sideways or upside-down this does not provide a proper data augmentation.
In somewhat of a response to your comment to Sammy’s post, the problem doesn’t restrict to five-degree rotations. The problem allows for all rotations.
An $8$ rotated 90 degrees is $\infty$ and no longer an $8$. Don’t train your neural net to see an $\infty$ and call it $8$.
A $6$ rotated 180 degrees is a $9$; a $9$ rotated 180 degrees is a $6$. Don’t train your neural net to mix up these. Unlike with $\infty$, there are $6$s and $9$s in your test data. If you allow for rotations of $6$s to $9$s and $9$s to $6$s, whatever improvements you see in being able to recognize the other digits may be offset by confusing $6$s and $9$s.
Similarly, consider a handwriting recognizer that confuses M and W.
“Dear Wom, Happy Wother’s Day!”
Or back to WNIST,$^{\dagger}$ “Dear Grandma, Happy 69th birthday! Next year we’ll go BASE jumping from the Burj Khalifa for your 100th!”
$^{\dagger}$Not a typo.
Sammy and Dave have accurately answered the question that Nielsen intended, but based on your comment on Sammy's answer I think you are wondering whether having arbitrarily many slightly-rotated training images would cause a problem.
Having arbitrarily many slightly-rotated would not be problematic; indeed, you could simulate this by generating a batch of images with small random linear transformations (rotation is a type of linear transformation) at each training step, and this approach is common. See for example this post
To just to add to the other answers here, there is nothing inherently wrong with putting arbitrarily-rotated images into a neural network. What Nielsen is hoping you will realise is the problem this would cause within the specific domain he's talking about (digit recognition): the problem isn't the rotation, the problem is telling the network that a 180-degree-rotated 9 is still a 9 (when actually it's a 6). If you were training a network to classify types of blood cell, arbitrary rotation of the training images would be a great strategy as the orientation of each cell wouldn't carry any special information.
