ANN training how to add
Foreword: I am currently studying ANN because I have ~ 18.5k images in ~ 83 classrooms. They will be used to train ANN to recognize roughly equal images in real time. I followed the example of the image in the book, but it doesn't work for me. So I go back to the beginning as I probably missed something.
I took the Encog XOR example and extended it to teach it how to add numbers less than 100. So far, the results are ambiguous, even for accurate input after training.
Inputs (normalized to 100) 0+0, 1+2, 3+4, 5+6, 7+8, 1+1, 2+2, 7.5+7.5, 7+7, 50+50, 20+20
. Outputs are numbers added and then normalized to 100.
After training 100,000 times, some sample is deduced from the input:
0+0=1E-18 (great!)
1+2=6.95
3+4=7.99 (so close!)
5+6=9.33
7+8=11.03
1+1=6.70
2+2=7.16
7.5+7.5=10.94
7+7=10.48
50+50=99.99 (woo!)
20+20=41.27 (close enough)
From invisible invisible data:
2+4=7.75 6+8=10.65 4+6=9.02 4+8=9.91 25+75=99.99 (!!) 21+21=87.41 (?)
I confused with layers, neuron numbers and [Resilient | Back] Propagation, but I'm not really sure if it will get better or worse. With the above data, the layers are 2, 6, 1.
I have no framework to judge this. This is normal? I don't have enough data entry? Is my data incomplete or random or too weighted?
source to share
You are not the first to ask this. It seems logical to teach ANN to add. We teach them to function as logic gates, why not add / multiply operators. I can't fully answer this question because I haven't researched it myself to see how well ANN works in this situation.
If you are just learning addition or multiplication, you can have better results with line out and no hidden layer. For example, to learn how to add, two weights must be 1.0, and the offset weight must be zero:
linear ((input1 * w1) + (input2 * w2) + offset) = becomes linear ((input1 * 1.0) + (input2 * 1.0) + (0.0)) =
Sigmoid or tanu training can be more problematic. The weight / bias and hidden layer would basically have to undo the sigmoid to revert to padding as described above.
I think part of the problem is that the neural network recognizes patterns rather than learning mathematics.
source to share
ANN can learn any function, including all arithmetic. For example, it has been proven that the addition of N numbers can be computed using a multidimensional network of depth 2. One way to teach NN arithmetic is to use a binary representation (i.e. not a normalized input of 100, but a set of input neurons, each of which represents one binary digit and the same representation for output). This way you can implement addition and other arithmetic. See this document for further discussion and description of ANN topologies used in teaching arithmetic.
PS. If you want to work with pattern recognition, it is not recommended to start practicing with your original dataset. Try some well-researched data such as MNIST where you know what results to expect from properly implemented algorithms. After mastering the classic examples, you can move on to working with your own data.
source to share
I'm in the middle of a demo that makes the computer learn how to propagate, and I share my progress on this: how Jeff suggested using a linear approach, and ADALINE in particular. At this point, my program "knows" how to multiply by 5. This is the result I get:
1 x 5 ~ = 5.17716232607829
2 x 5 ~ = 10.147218373698
3 x 5 ~ = 15.1172744213176
4 x 5 ~ = 20.0873304689373
5 x 5 ~ = 25.057386516557
6 x 5 ~ = 30.0274425641767
7 x 5 ~ = 34.9974986117963
8 x 5 ~ = 39.967554659416
9 x 5 ~ = 44.9376107070357
10 x 5 ~ = 49.9076667546553
Let me know if you are interested in this demo. I would be happy to share.
source to share