I've implemented simple Widrow-Hopf perceptron and k-means clustering and compare results against MNIST data set. I didn't expect great results because of linear nature of these algorithms. WH perceptron ended up with ~70% accuracy and k-means with ~50% (I have unit tests and compare with pure random chance 1/10 I suppose they do work somehow correct).
Also I gave "a hint" to k-means by set up initial centroids around different digits, so it can converge faster.
I suppose that k-means showed this result because digits are pretty much similar (6 is like 8 and 8 is like 9). Then I assumed that generalizations ability of WH perceptron is better than k-means. Did anyone see any article/book about this topic? I want to understand results from a rigor mathematical perspective.