4
$\begingroup$

In this nice tutorial about CNNs, the authors build a single-layer CNN. The initial convolution weights are set randomly, according to a uniform distribution.

By the end of this scetion, the authors note that the randomly initialised kernel behaves very similar to an edge detector and give the following input and output as example.

In/Output

Why does the randomly initialised kernel behave like an edge detector?

$\endgroup$
3
$\begingroup$

Take this with a grain of salt, but I think this is simply not true. You can evaluate it with the code I just wrote:

enter image description here

Only 2, probably 3 of the 25 look like edge filters to me. The result of an edge filter looks like this:

enter image description here

| improve this answer | |
$\endgroup$
  • $\begingroup$ This is a very bad answer... All the results that you just showed are very similar with the following differences: - Some have a flipped sign, meaning that the light edges become dark - Some have a constant that is added to them. However, the overall behaviour is similar to an edge detector. Your images just prove that the original question is right. $\endgroup$ – DomDev Jan 31 '19 at 5:08
  • $\begingroup$ "All the results that you just showed are very similar " - that is exactly what I wanted to show :-) $\endgroup$ – Martin Thoma Jan 31 '19 at 5:54
  • $\begingroup$ "However, the overall behavior is similar to an edge detector." - I don't think that the top images look similar to the bottom image. $\endgroup$ – Martin Thoma Jan 31 '19 at 5:55
  • $\begingroup$ Because your bottom image is a vertical edge detector. If you compute the horizontal edge detector, then compute their norm, it will be very similar to the other images (with an added bias and a possible negative factor). The reason is that the random initialization will obviously not be biased toward a specific direction. $\endgroup$ – DomDev Feb 17 '19 at 15:06
  • $\begingroup$ "the random initialization will obviously not be biased toward a specific direction" - that is exactly what my answer shows $\endgroup$ – Martin Thoma Feb 17 '19 at 20:22
0
$\begingroup$

The answer by Martin is not right. In his examples, he only shows that the deep network does not behave like a directional edge detector. In fact, his example below is a smoothed vertical derivative (probably Sobel).

However, we clearly see that it behaves like a direction-agnostic edge detector, such as a Laplacian or the norm of a gradient. Also, we can see that all images look the same, with the only major differences being that some are inverted and others have an additional constant.

In summary, the network behaves like an direction-agnostic edge detector. A bad edge detector, but still an edge detector. This helps explain why deep learning edge detection methods converge a lot faster and with simpler architectures than other problems such as saliency and skeleton extraction.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.