2
$\begingroup$

Assume I have a CNN that in the first (conv) layer takes a 1-channel signal (the input) and gives a 2-channel output. Let's further assume that the rest of the net has symmetric architecture from the point-of-view of any of those channels. Moreover, let's assume all the weights are initialized by zero (or the same number).

How do we know that the feature maps (kernels of the channels) after training are not similar given the symmetry in such network? Specifically, what is unreasonable to have kernels with identical feature maps?

$\endgroup$

1 Answer 1

2
$\begingroup$

You shouldn't initialize all the weights with zero, use random initialization instead. Otherwise you can't break the symmetry and all the outputs in the network would be same.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.