Correct method to combine Attention, Multi, Remaining cell wrappers in 1.1.0-rc2
I am trying to combine the following:
tf.contrib.rnn.AttentionCellWrapper
tf.contrib.rnn.MultiRNNCell
tf.contrib.rnn.ResidualWrapper
tf.contrib.rnn.LSTMCell
I am building a cell like this
cell = tf.contrib.rnn.AttentionCellWrapper(
tf.contrib.rnn.MultiRNNCell([
tf.contrib.rnn.ResidualWrapper(
cell=tf.contrib.rnn.LSTMCell(dec_units))
for _ in range(dec_layers)]),
attn_length=attn_len)
This works great if I keep attn_len small (1-2), but increasing attn_len to a larger value (5+) makes the script hang indefinitely with one CPU core tied to 100% at the start of training (0 steps completed).
Is this an appropriate way to combine these elements? Should I override the default values ββfor optional parameters?
+3
source to share
No one has answered this question yet
Check out similar questions: