-1
$\begingroup$

In convolutional neutral network, the weights are shared within a feature map. What about two different feature map? How to make them different (so that we don't learn the same thing again).

Q: What exactly in the training algorithm to make it so that the weights are different across different feature maps. For example, if I define 2 feature maps, does the network guarantee that the weights are different in feature map A and feature map B?

$\endgroup$

1 Answer 1

0
$\begingroup$

By randomizing your initial weights, the gradients will flow differently through the network. If all the feature maps would converge to the same weight set, there is much less to learn and your loss will be higher than learning different features. Let's say they start very, very close to each other, then making them more different from eachother makes the network more expressive. Due to the nature of backpropagation these weights will diverge as opposed to converge (in general).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.