机器学习的一个简单例子

It is a very simple kind of NeuralNet and it is gonna be learning to recognize digits and you gonna be able to see how the weights  evolved as we run a very simple learning algorithm.

A very simple learning algorithm for traing a very simple network to recognize handwritten shapes. The network has two layers of neurons. It has input neuron whose activities represent the intensity of pixels, and output neurons, whose activities represent the class.

Consider a neural network with two layers of neurons.

– neurons in the top layer represent known shapes.

– neurons in the bottom layer represent pixel intensities.

What we'd like is that when we show a paritcular shape. The output neuron for that shape get active. If a pixel is active what it does is its votes for particular shapes.Namely the shapes that contain that pixel.

• A pixel gets to vote if it has ink on it.

– Each inked pixel can vote for several different shapes.

• The shape that gets the most votes wins.

第一步：我们需要决定如何显示权值,it seems natural to write the weights on the connection between input unit and output unit. But , we are never to be able to see what was going on if we get that.We need a display in which we can see the values of thousands of weghts. So the idea is for each output unit, we make a little map. And int that map we show the strenght of connection coming from each input pixel in the location of that input pixel. And we show the strength of connection by using 

a black or white blob, whose  area represents the magnitude of the weight and the color representing the sign. So the initial weights that you see there are just small random weights

Now what we gonna do is show that network some data and get it to learn weights that are better than the random weights.

the way we are gonna look is when we show it an image, we are going to increment the weights from the active pixels in the image to the correct class. 

If we just did that, the weights could get only bigger and eventually every class , so we need some way of keeping the weights under control. 

What we gonna do is we will also gonna decrement weights from the active pixels to whatever class the network guesses.

初始化的权值：

训练中：

 

 

 

最终：

看每个单元的权值，有点像数字模板

Why the simple learning algorithm is insufficient

• A two layer network with a single winner in the top layer is equivalent to having a rigid template for each shape.

– The winner is the template that has the biggest overlap with the ink.

• The ways in which hand-written digits vary are much too complicated to be captured by simple template matches of whole shapes.

– To capture all the allowable variations of a digit we need to learn the features that it is composed of