How to get scales from tensorflow fully_linked
I am trying to extract weights from a model after training. Here is an example of a toy
import tensorflow as tf
import numpy as np
X_ = tf.placeholder(tf.float64, [None, 5], name="Input")
Y_ = tf.placeholder(tf.float64, [None, 1], name="Output")
X = ...
Y = ...
with tf.name_scope("LogReg"):
pred = fully_connected(X_, 1, activation_fn=tf.nn.sigmoid)
loss = tf.losses.mean_squared_error(labels=Y_, predictions=pred)
training_ops = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(200):
sess.run(training_ops, feed_dict={
X_: X,
Y_: Y
})
if (i + 1) % 100 == 0:
print("Accuracy: ", sess.run(accuracy, feed_dict={
X_: X,
Y_: Y
}))
# Get weights of *pred* here
I have looked at Get weights from tensorflow model and docs but can't find a way to get the weights value.
So, in the toy example, suppose X_ has shape (1000, 5), how can I get 5 values ββin weights of 1 layer after
source to share
There are some issues in your code that need to be fixed:
1- You need to use variable_scope
instead name_scope
in the following line (see TensorFlow documentation for the difference between the two):
with tf.name_scope("LogReg"):
2- To be able to get a variable later in the code, you need to know its name. So, you need to assign a name to the variable of interest (if you don't support it, it will be assigned by default, but then you need to figure out what it is!):
pred = tf.contrib.layers.fully_connected(X_, 1, activation_fn=tf.nn.sigmoid, scope = 'fc1')
Now let's see how the above fixes can help us get the value of a variable. Each layer has two types of variables: weights and biases. In the following code snippet (your modified version), I will only show you how to extract the weight for a fully connected layer:
X_ = tf.placeholder(tf.float64, [None, 5], name="Input")
Y_ = tf.placeholder(tf.float64, [None, 1], name="Output")
X = np.random.randint(1,10,[10,5])
Y = np.random.randint(0,2,[10,1])
with tf.variable_scope("LogReg"):
pred = tf.fully_connected(X_, 1, activation_fn=tf.nn.sigmoid, scope = 'fc1')
loss = tf.losses.mean_squared_error(labels=Y_, predictions=pred)
training_ops = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
with tf.Session() as sess:
all_vars= tf.global_variables()
def get_var(name):
for i in range(len(all_vars)):
if all_vars[i].name.startswith(name):
return all_vars[i]
return None
fc1_var = get_var('LogReg/fc1/weights')
sess.run(tf.global_variables_initializer())
for i in range(200):
_,fc1_var_np = sess.run([training_ops,fc1_var], feed_dict={
X_: X,
Y_: Y
})
print fc1_var_np
source to share
Try the following:
with tf.Session() as sess:
last_check = tf.train.latest_checkpoint(tf_data)
saver = tf.train.import_meta_graph(last_check+'.meta')
saver.restore(sess,last_check)
######
Model_variables = tf.GraphKeys.MODEL_VARIABLES
Global_Variables = tf.GraphKeys.GLOBAL_VARIABLES
######
all_vars = tf.get_collection(Model_variables)
# print (all_vars)
for i in all_vars:
print (str(i) + ' --> '+ str(i.eval()))
I got it:
<tf.Variable 'linear/linear_model/DOLocationID/weights/part_0:0' shape=(1, 1) dtype=float32_ref> --> [[-0.00912262]]
<tf.Variable 'linear/linear_model/PULocationID/weights/part_0:0' shape=(1, 1) dtype=float32_ref> --> [[ 0.00573495]]
<tf.Variable 'linear/linear_model/passenger_count/weights/part_0:0' shape=(1, 1) dtype=float32_ref> --> [[-0.07072949]]
<tf.Variable 'linear/linear_model/trip_distance/weights/part_0:0' shape=(1, 1) dtype=float32_ref> --> [[ 2.59973669]]
<tf.Variable 'linear/linear_model/bias_weights/part_0:0' shape=(1,) dtype=float32_ref> --> [ 4.27982235]
source to share