Parameter values in RNNlib
I am new to learning repetitive neural networks and am now confused about the parameters in RNNLib. In particular, I don't understand hidden block, hidden size, input block, sub-task size, and so on with mdl. In my experience, I only had input vectors, one hidden lstm layer, and a softmax output level. Why does the block look like a matrix?
source to share
RNNLib implements a new type of RNN, the so-called "multidimensional recurrent neural network". The following link on the RNNLib page explains that: Alex Graves, Santiago Fernandez and Jurgen Schmidhuber. Multidimensional Repeated Neural Networks International Conference on Artificial Neural Networks, September 2007, Port. This extension is intended for processing images, videos, etc. As the doc explains:
"The basic idea of MDRNN is to replace the only recurrent join found in a standard RNN with as many reconnections as the data sizes. During the forward pass at each point in the data sequence, the hidden layer of the network receives both an external input and its own activations from a single step back all sizes "
I think this is why you have the option to use multidimensional input. If you want to use the RNNLib like a normal 1D RNN, just specify one dimension for the input and the LSTM block.
MDL stands for Minimum Description Length cost function used to approximate Bayesian inference (NN regularization method). If you want to use this, your best bet is to read the original links provided on the RNNLib website. Otherwise, I think it can be simply ignored.
source to share