Initializing Variables in Unified Tensor Graphs

I am trying to combine multiple Tensorflow models on separate plots into a single model on a single plot (I assume typical scenarios are learning transfer models, ensembles, etc. that need to be frozen).

I can connect two plots using tf.import_graph_def. However, when one of the original plots contains a variable, I cannot insert values ​​into it in the new plot. The assignments are copied as part of the graph, and if I run them manually, I get the expected results. (Let's uncomment the sess.run

-channel in the fourth with graph...

-block)

import tensorflow as tf

graph1 = tf.Graph()
with graph1.as_default():
    with tf.Session(graph=graph1) as sess:
        var1 = tf.Variable([1, 1, 1], name='var1')
        tensor1 = tf.multiply(var1, 2, name='tensor1')

graph2 = tf.Graph()
with graph2.as_default():
    with tf.Session(graph=graph2) as sess:
        placeholder = tf.placeholder(tf.int32, [3], name='placeholder')
        tensor2 = tf.multiply(placeholder, 2, name='tensor2')

graph3 = tf.Graph()
with graph3.as_default():
    with tf.Session(graph=graph3) as sess:
        graph1out, = tf.import_graph_def(graph1.as_graph_def(), name='graph1', return_elements=['tensor1:0'])
        tf.import_graph_def(graph2.as_graph_def(), name='graph2', input_map={'placeholder:0': graph1out})

with graph3.as_default():
    with tf.Session(graph=graph3) as sess:
        sess.run(tf.global_variables_initializer())
        #sess.run(graph3.get_tensor_by_name('graph1/var1/Assign:0'))
        tensor = graph3.get_tensor_by_name('graph2/tensor2:0')
        result = sess.run(tensor)

      

Running this snippet as-is gives:

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value graph1/var1

      

In manual mode, assign-ops is a very small generic approach that would be messy to write as part of a generic function. It also requires that these destination calls be made every time I reinitialize the session with the graph.

Is there a more general way to do this? For example, creating new variables based on old ones and binding them to an existing structure?

If not, is there a way to wrap the destination calls? So at least I can run a single generic init operation instead of an unknown number specific for each case?

+3


source to share


1 answer


Fixed this by creating init-wrapper. For all tensors with names ending with "Apply: (int)" I am using tf.reduce_sum (to standardize their shapes) and stuck them in tf.stack. This initialization operation is internally treated as a noop, but ensures that every tensor that is passed to it gets triggered (and therefore also initializes their corresponding variable). Semantically, this can be thought of as tf.global_variables_initializer ().



Not the most elegant solution, but it works for my applications so far.

-1


source







All Articles