Initializing Variables in Unified Tensor Graphs
I am trying to combine multiple Tensorflow models on separate plots into a single model on a single plot (I assume typical scenarios are learning transfer models, ensembles, etc. that need to be frozen).
I can connect two plots using tf.import_graph_def. However, when one of the original plots contains a variable, I cannot insert values into it in the new plot. The assignments are copied as part of the graph, and if I run them manually, I get the expected results. (Let's uncomment the sess.run
-channel in the fourth with graph...
-block)
import tensorflow as tf
graph1 = tf.Graph()
with graph1.as_default():
with tf.Session(graph=graph1) as sess:
var1 = tf.Variable([1, 1, 1], name='var1')
tensor1 = tf.multiply(var1, 2, name='tensor1')
graph2 = tf.Graph()
with graph2.as_default():
with tf.Session(graph=graph2) as sess:
placeholder = tf.placeholder(tf.int32, [3], name='placeholder')
tensor2 = tf.multiply(placeholder, 2, name='tensor2')
graph3 = tf.Graph()
with graph3.as_default():
with tf.Session(graph=graph3) as sess:
graph1out, = tf.import_graph_def(graph1.as_graph_def(), name='graph1', return_elements=['tensor1:0'])
tf.import_graph_def(graph2.as_graph_def(), name='graph2', input_map={'placeholder:0': graph1out})
with graph3.as_default():
with tf.Session(graph=graph3) as sess:
sess.run(tf.global_variables_initializer())
#sess.run(graph3.get_tensor_by_name('graph1/var1/Assign:0'))
tensor = graph3.get_tensor_by_name('graph2/tensor2:0')
result = sess.run(tensor)
Running this snippet as-is gives:
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value graph1/var1
In manual mode, assign-ops is a very small generic approach that would be messy to write as part of a generic function. It also requires that these destination calls be made every time I reinitialize the session with the graph.
Is there a more general way to do this? For example, creating new variables based on old ones and binding them to an existing structure?
If not, is there a way to wrap the destination calls? So at least I can run a single generic init operation instead of an unknown number specific for each case?
source to share
Fixed this by creating init-wrapper. For all tensors with names ending with "Apply: (int)" I am using tf.reduce_sum (to standardize their shapes) and stuck them in tf.stack. This initialization operation is internally treated as a noop, but ensures that every tensor that is passed to it gets triggered (and therefore also initializes their corresponding variable). Semantically, this can be thought of as tf.global_variables_initializer ().
Not the most elegant solution, but it works for my applications so far.
source to share