Depth of artificial neural networks
According to this answer, you should never use more than two hidden layers of neurons.
According to this answer, the middle layer should contain no more than twice the number of input or output neurons (so if you have 5 input neurons and 10 output neurons, you should use (no more) 20 middle neurons per layer).
Does this mean that all data will be modeled within this number of neurons?
So, if, for example, someone wants to do something, simulating the weather (a million input nodes from data from different weather stations) to a simple OCR (scanned text with 1000x1000DPI resolution), the same number of nodes would be required
PS.
My last question was closed. Is there another SE site that discusses these issues?
You will likely recycle your data (eg High Variance). Think of it this way: the more neurons and layers you have, the more parameters that better fit your data.
Remember for the first layer node the equation becomes Z = sigmoid (sum (W * x)) The second layer node becomes Z2 = Sigmoid (sum (W * Z))
Take a look at the machine learning class taught at Stanford ... its a great online course and a good tool as a reference.
source to share
In some architectures, more than two hidden layers may be useful, such as cascading correlation (Fahlman and Lebiere 1990) and in special applications such as the double spiral problem (Lang and Witbrock 1988) and zip recognition (Le Cun et al., 1989) ...
- Fahlman, SE and Lebiere, C. (1990), "Cascade Correlation Architecture for Learning," NIPS2, 524-532.
- Le Cun, Y., Boser, B., Denker, Js, Henderson, D., Howard, RE, Hubbard, W. and Jackel, L.D. (1989), “Backpropagation of handwritten zip code recognition,” Neural Computation, 1, 541-551.
Check out the "How many hidden layers should I use?" and "How many hidden units should I use?" See comp.ai.neural-nets FAQ for more information.
source to share