k-means and MNIST dataset

Just thought few days ago how should look avarage 0 or 1 or 9 or any number. So downloaded MNIST dataset and implemented k-means algorithm (I could calculate average vector in every cluster, but that wont be interesting).

I already had ultra-simple implementation of the perceptron and was interesting to compare them. Perceptron has success rate around ~70% (not bad for this ancient approach) and k-means had ~50%. I was wondering why it's so bad and have plotted estimated centroids (recall that centroids are actually vector in the same vector space, so if I have pictures of digits as vectors, centroid is a picture as well). So I gave "a hint" to k-means by setting first gues as a vectors from each cluster, so solution would converge faster.

On of experiments showed

Quite interesting how 9 cluster pulled 4th centroid, the same with 7 and 9. I started to search around for an answer and found it in the stackexchange. Look in the mouse, how k-means centroid pulled some points.

So, next step is to implement EM-clustering and compare performance.

I've used self developed k-means implementaion (some sort of library function with numpy dependency), and mnist client code, also code to estimate performance and plot centroids.