def objective_L2(dst): # Our training objective. Google has since release a way to load dst.diff[:] = dst.data # arbitrary objectives from other images. We'll go into this later. def make_step(net, step_size=1.5, end='inception_4c/output', jitter=32, clip=True, objective=objective_L2): '''Basic gradient ascent step.''' src = net.blobs['data'] # input image is stored in Net's 'data' blob dst = net.blobs[end] ox, oy = np.random.randint(-jitter, jitter+1, 2) src.data = np.roll(np.roll(src.data, ox, -1), oy, -2) # apply jitter shift net.forward(end=end) objective(dst) # specify the optimization objective net.backward(start=end) g = src.diff # apply normalized ascent step to the input image src.data[:] += step_size/np.abs(g).mean() * g src.data = np.roll(np.roll(src.data, -ox, -1), -oy, -2) # unshift image if clip: bias = net.transformer.mean['data'] src.data[:] = np.clip(src.data, -bias, 255-bias)
If I understand what is going on correctly, the input image in
net.blobs['data'] is inserted into the NN until the layer
end. Once, the forward pass is complete until
end, it calculates how "off" the blob at the
end is from "something".
What is this "something"? Is it
dst.data? I stepped through a debugger and found that
dst.datawas just a matrix of zeros right after the assignment and then filled with values after the backward pass.
Anyways, assuming it finds how "off" the result of the forward pass is, why does it try to do a backwards propagation? I thought the point of deep dream wasn't to further train the model but "morph" the input image into whatever the original model's layer represents.
What exactly does
src.data[:] += step_size/np.abs(g).mean() * gdo? It seems like applying whatever calculation was done above to the original image. Is this line what actually "morphs" the image?
Links that I have already read through
I would be interested in what the author of the accepted answer meant by
we take the original layer blob and "enchance" signals in it. What does it mean, I don't know. Maybe they just multiply the values by coefficient, maybe something else.
In this blog post the author comments next to
src.data[:] += step_size/np.abs(g).mean() * g: "get closer to our target data." I'm not too clear what "target data" means here.
Note I'm cross posting this from https://stackoverflow.com/q/40690099/2750819 as I was recommended to in a comment.