Octave online

Guys, have you seen Octave online? quite impressive thing, you can store!!! your scripts, it can plot and does symbolic evaluations! Basic Machine Learning, Linear Algebra, Convex optimization, etc courses can be done right without any investment in an expensive software (hello Matlab basic bundle for 5k). w00h00

http://octave-online.net/

SVD low rank approximation for pictures (SVD image compression on Python3)

Well, tons of posts were written about it. It wont be "yet another one". But I implemented it anyway, just for fun. I wont explain theoretical part, but will suggest some links about it.

So good docs are:
http://timbaumann.info/svd-image-compression-demo/

Some good docs (thx Berkeley). This book is abandoned now, but it's good to read and store link for future reference.
http://inst.eecs.berkeley.edu/~ee127a/book/login/l_svd_low_rank.html
http://inst.eecs.berkeley.edu/~ee127a/book/login/l_svd_apps_image.html

Just nice presentation about PCA and SVD
http://math.arizona.edu/~brio/VIGRE/ThursdayTalk.pdf

and the source svd-img-compression.py

magic happens in np.dot() we pickup just take approx_rank singular values and discard others, so get some compression.

Result is below, on left side depicted original greyscale picture with rank equals 440 on a right side depicted low rank approximation with rank equals 50

Tensorflow in Gentoo virtualenv

Just tried to install tensorflow in Gentoo virtual env with python 3.4 and failed

k-means and MNIST dataset

Just thought few days ago how should look avarage 0 or 1 or 9 or any number. So downloaded MNIST dataset and implemented k-means algorithm (I could calculate average vector in every cluster, but that wont be interesting).

Two good books about neuro networks

Long time no see :) short post about machine learning. I wanted to go deeper into this field and tried to find some good book, not like a "Machine Learning in 24 hours" but something grad level with acceptable science payload. So I've read ~30% of Simon Haykin "Neural Networks and Learning Machines (3rd Edition)". Book is good, but not enough naive examples to jump in, like 1 neuron with 2 inputs, but science payload there is nice. Gazzillion links to articles and books, so book is useful when you know what to do. Then I found nice complementary book with naive examples. Also it's really cheap on Amazon - about 25$, also it can be downloaded for free from the official site. "Neural Network Design (2nd Edition)" by Hagan, et al. So have fun.

Meanwhile, I started uber small project - neural networks without any special lib. I use only numpy for matrix calculation. Everything slow but primitive and self explanatory. As a data set I'm using MNIST data set, it has 60k training samples and 10k test samples.

Simple implementation of Widrow-Hoff perceptron is here, ~70% success rate after training
https://github.com/venik/simple_neuro_networks/tree/master/src/one_layer_mnist

And MNIST reader is here
https://github.com/venik/simple_neuro_networks/blob/master/lib/mnist/mnist.py

also you probably want to see how does MNIST samples look on a screen
https://github.com/venik/simple_neuro_networks/tree/master/utils/mnist_reader

How to download MNIST is here
https://github.com/venik/simple_neuro_networks/tree/master/data_set/mnist