Clever Geek Handbook

Keras get_weight for RNN

When I run this code with Keras:

networkDrive = Input(batch_shape=(1,length,1))
network = SimpleRNN(3, activation='tanh', stateful=False, return_sequences=True)(networkDrive)

generatorNetwork = Model(networkDrive, network)

predictions = generatorNetwork.predict(noInput, batch_size=length)


print(np.array(generatorNetwork.layers[1].get_weights()))

I am getting this output

[array([[ 0.91814435,  0.2490257 ,  1.09242284]], dtype=float32)
 array([[-0.42028981,  0.68996912, -0.58932084],
       [-0.88647962, -0.17359462,  0.42897415],
       [ 0.19367599,  0.70271438,  0.68460363]], dtype=float32)
 array([ 0.,  0.,  0.], dtype=float32)]

I suppose matrix (3,3) is a weight matrix connecting RNN units to each other and one of the two arrays is probably an offset. But what is the third one?

+3


source to share


1 answer


The simpleRNN implementation actually requires 3 sets of weights.

weights[0]

is the input matrix. It transforms the input and therefore has the form[input_dim, output_dim]

weights[1]

is the recurrence matrix. It transforms the recurrent state and has the form[output_dim, output_dim]



weights[2]

- offset matrix. It is added to the output and has the form[output_dim]

the results of the three operations are added together and then passed through the activation level.

Hope it's clear now?

+1


source






More articles:


All Articles