# Why does Siamese neural networks use tied weights and how do they work?

Reading this paper on one-shot learning "Siamese Neural Networks for One-shot Image Recognition" I was introduced to the idea of Siamese Neural Networks.

What I did not fully grasp was what they meant by this line:

This objective is combined with standard
across the twin networks due to the tied weights.

Firstly, how exactly are they tied? Bellow, I believe I've provided the formula by which they compute the gradient. T is the epoch, $$\mu_j$$ is the momentum, $$\lambda_j$$ the regularization, $$\eta_j$$ the learning rate, $$w_{kj}$$ I believe to be the weight between neuron k and in one layer and j in another but correct me if I'm wrong.

$$$$\begin{array}{c} \mathbf{w}_{k j}^{(T)}\left(x_{1}^{(i)}, x_{2}^{(i)}\right)=\mathbf{w}_{k j}^{(T)}+\Delta \mathbf{w}_{k j}^{(T)}\left(x_{1}^{(i)}, x_{2}^{(i)}\right)+2 \lambda_{j}\left|\mathbf{w}_{k j}\right| \\ \Delta \mathbf{w}_{k j}^{(T)}\left(x_{1}^{(i)}, x_{2}^{(i)}\right)=-\eta_{j} \nabla w_{k j}^{(T)}+\mu_{j} \Delta \mathbf{w}_{k j}^{(T-1)} \end{array}$$$$

My other question is why this is even desirable? Why not just reuse the same network twice? Or perhaps the two networks will be identical after training? If the networks are identical after training, why would you set it up like this? What benefits does it have?

• Have a look at this post: datascience.stackexchange.com/a/43820/98414 which should provide the intuition behind it. Jun 6, 2020 at 13:21
• Thanks for the link! In a Siamese network, the weights are identical on both branches. In the paper I was reading the weights are kept similar by computing the combined gradient for both sides and using that. Though I'm not sure how they do with initialization of the network. If both branches are initialized to different values the trained branches should differ (but probably not to much). Jun 7, 2020 at 11:32
• I doubt they are initialized differently. They should share the same weights. Jun 7, 2020 at 12:04