Implementation of the article "Deep inside convolutional networks: visualization of image classification models and significance maps", Simonyan et al.

Question

Implementation of the article "Deep inside convolutional networks: visualization of image classification models and significance maps", Simonyan et al.

When rendering gradient data in convolutional neural networks using the Caffe framework, having already rendered gradient data across all classes, it is interesting to take the gradient in relation to a specific class. in the deploy.prototxt file in the "bvlc_reference_caffenet" model, I have set:

force_backward: true

and commented the last part:

layer {
  name: "prob" 
  type: "Softmax"
  bottom: "fc8"
  top: "prob" 
}

which is before:

layer {
  name: "fc8"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc8"
  inner_product_param {
  num_output: 1000
 }
}

and it adds:

 layer {
   name: "loss"
   type: "SoftmaxWithLoss"
   bottom: "fc8"
   bottom: "label"
   top: "prob"
}

in python code by calling:

out = net.forward()

we go to the last layer and then by calling:

backout = net.backward()

got a render of the gradient. First, I would like to ask what is called a significance map, and if I want to do backward with regard to a specific class, for example. 281 for a cat. what should I do?

in advance for guidance.

PS used Yangqing code for my laptop in filter rendering.

imagenetMeanFile = caffe_root  +'python/caffe/imagenet/ilsvrc_2012_mean.npy'
caffe.set_mode_cpu()
net = caffe.Net(caffe_root +   'models/bvlc_reference_caffenet/deploy.prototxt',
            caffe_root + 'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel',
            caffe.TRAIN)
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2,0,1))
transformer.set_mean('data', np.load(caffe_root + 'python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1)) # mean pixel 
transformer.set_raw_scale('data', 255)  # the reference model operates on images in [0,255] range instead of [0,1]
transformer.set_channel_swap('data', (2,1,0))  # the reference model has channels in BGR order instead of RGB

+3

python deep-learning gradient caffe visualization

Majid azimi 28 june 15 at 22:46

source to share

2 answers

using the following code, it can be done:

label_index = 281  # Index for cat class
caffe_data = np.random.random((1,3,227,227))
caffeLabel = np.zeros((1,1000,1,1))
caffeLabel[0,label_index,0,0] = 1;

bw = net.backward(**{net.outputs[0]: caffeLabel})

+1

Majid azimi 13 jul. 15 at 12:49 am

source to share

Majid azimi · Accepted Answer · 2015-08-28T13:06:36+0000

also for full visualization you can refer to my github which is more complete and render the significance map as well as class model rendering and gradient rendering in backpropagagation.

https://github.com/smajida/Deep_Inside_Convolutional_Networks

Implementation of the article "Deep inside convolutional networks: visualization of image classification models and significance maps", Simonyan et al.

More articles: