Clever Geek Handbook

Tensorflow gradients: no automatic implicit sum

In tensorflow, if one has two tensors x

and y

, and needs to have gradients y

with respect to x

using tf.gradients(y,x)

. Then it actually turns out:

gradient[n,m] = sum_ij d y[i,j]/ d x[n,m]

There is a sum over the y indices, is there a way to avoid this implicit sum? To get the whole gradient tensor gradient[i,j,n,m]

?

+4


source to share


2 answers


Here's my work, just taking the derivative of each component (as mentioned also by @ Yaroslav) and then putting them back together again in the case of rank 2 tensors (Matrices):

import tensorflow as tf

def twodtensor2list(tensor,m,n):
    s = [[tf.slice(tensor,[j,i],[1,1]) for i in range(n)] for j in range(m)]
    fs = []
    for l in s:
        fs.extend(l)
    return fs

def grads_all_comp(y, shapey, x, shapex):
    yl = twodtensor2list(y,shapey[0],shapey[1])
    grads = [tf.gradients(yle,x)[0] for yle in yl]
    gradsp = tf.pack(grads)
    gradst = tf.reshape(gradsp,shape=(shapey[0],shapey[1],shapex[0],shapex[1]))
    return gradst


Now grads_all_comp(y, shapey, x, shapex)

will output a tensor of rank 4 in the desired format. This is a very inefficient way because everything needs to be sliced ​​and repackaged together, so if anyone finds a better one, I will be very interested to see it.

+4


source


There is no way. TensorFlow 0.11 tf.gradients

implements the standard inverse AD mode, which gives a scalar derivative. You will need to call tf.gradients

for each y[i,j]

separately



+3


source






More articles:


All Articles