Let's say I start with a standard conv net architecture capable of 99% accuracy on MNIST such as this one, but let's say I merge in an "easy" feature to the fully-connecter layer such as a vector of length 10 that encodes the correct output digit 95% of the time and a random digit otherwise.
- Will my network ever reach 99% accuracy
- Will it take longer to get there?
I think the answer the question 1 is yes, because we will should always be able to find a downhill in the error before the conv net path has reached its full potential. In fact I think it may surpass the accuracy of the original architecture since we are leaking information about the correct output labels.
However I am not so sure about question 2. Does this make the shape of error function unfavorable in any way? I can't decide whether reaching the same accuracy of the original architecture will be faster, slower, or exactly the same.
Hopefully there is an easy answer, otherwise I will run an experiment and report back!