Deep Dream code does not generate recognizable patterns
I tried to create my own Deep Dream algorithm with this code:
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import inception
img = np.random.rand(1,500,500,3)
net = inception.get_inception_model()
tf.import_graph_def(net['graph_def'], name='inception')
graph = tf.get_default_graph()
sess = tf.Session()
layer = graph.get_tensor_by_name('inception/mixed5b_pool_reduce_pre_relu:0')
gradient = tf.gradients(tf.reduce_mean(layer), graph.get_tensor_by_name('inception/input:0'))
softmax = sess.graph.get_tensor_by_name('inception/softmax2:0')
iters = 100
init = tf.global_variables_initializer()
sess.run(init)
for i in range(iters):
prediction = sess.run(softmax, \
{'inception/input:0': img})
grad = sess.run(gradient[0], \
{'inception/input:0': img})
grad = (grad-np.mean(grad))/np.std(grad)
img = grad
plt.imshow(img[0])
plt.savefig('output/'+str(i+1)+'.png')
plt.close('all')
But even after running this loop for 100 iterations, the resulting image still looks random (I'll attach the above image to this Question).
Can anyone help me optimize my code?
source to share
Using the initial network for Deep Dream is a bit tricky. In the CADL course from which you borrowed the supporting library, the instructor prefers to use VGG16 instead. If you use this and make a few small changes to your code, you should end up with something that works (if you change the location on the initial network here, it will do some work, but the results will look even more disappointing):
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import vgg16 as vgg
# Note reduced range of image, your noise function was drowning
# out the few textures that you were getting
img = np.random.rand(1,500,500,3) * 0.1 + 0.45
net = vgg.get_vgg_model()
tf.import_graph_def(net['graph_def'], name='vgg')
graph = tf.get_default_graph()
sess = tf.Session()
layer = graph.get_tensor_by_name('vgg/pool4:0')
gradient = tf.gradients(tf.reduce_mean(layer),
graph.get_tensor_by_name('vgg/images:0'))
# You don't need to define or use the softmax layer - TensorFlow
# is smart enough to resolve the computation graph for gradients
# without explicitly running the whole network forward first
iters = 100
# You don't need to init the network variables, everything you need
# is set by the import, plus the placeholder.
for i in range(iters):
grad = sess.run(gradient[0], {'vgg/images:0': img})
# You can use all sorts of normalisation, this one is from CADL
grad /= (np.max(np.abs(grad))+1e-7)
# You forgot to use += here, and it is best to use a
# step size even after gradient normalisation
img += 0.25 * grad
# Re-normalise the image, to prevent over-saturation
img = 0.98 * (img - 0.5) + 0.5
img = np.clip(img, 0.0, 1.0)
plt.imshow(img[0])
plt.savefig('output/'+str(i+1)+'.png')
plt.close('all')
print(i)
Doing all of this creates images that clearly work, but still need some tweaking:
To get better, full color images of the type you may have seen on the internet require more changes. For example, you could normalize or slightly blur the image between each iteration.
If you want to get a little more sophisticated, you can try the TensorFlow Jupyter laptop walk , although it is somewhat harder to grasp from first principles due to the combination of several ideas.
source to share