# How gradients are flown back to Network in siamese architecture? How weights of all CNN models are same even when using different models

TL;DR : Intuition behind the gradient flow in Siamese Network? How can 3 models share same weights? And if 1 model is used, hpw Gradients are updated from 3 different paths?

I am trying to build to a Siamese Network and as far as I can know, if I have to build a Triplet Loss based Siamese, I have to use 3 different networks. So for simplicity, let us say that my architecture is something like: Please correct the architecture if wrong

    I1 = Input(shape=image_shape)
I2 = Input(shape=image_shape)
I3 = Input(shape=image_shape)

res_m_1 = ResNet50(include_top=False, weights='imagenet', input_tensor=I1, pooling='avg')
res_m_2 = ResNet50(include_top=False, weights='imagenet', input_tensor=I2, pooling='avg')
res_m_3 = ResNet50(include_top=False, weights='imagenet', input_tensor=I3, pooling='avg')

x1 = res_m_1.output
x2 = res_m_2.output
x3 = res_m_3.output

# x = Flatten()(x) or use this one if not using any pooling layer

##### ------- ---------------------------- --------- ########

'NEED HELP AFTER THIS ONE; HOW TO BUILD ARCHITECTURE'

########### ------------------------------------ ###########

siamese_model = Model(inputs=[I1,I2], outputs=final_output)

siamese_model.compile(loss=some_triplet_loss,optimizer=Adam(),metrics['acc'])

siamese_model.fit_generator(train_gen,steps_per_epoch=1000,epochs=10,validation_data=validation_data)


My Understanding and Question:: If there are 3 networks in the architecture, how come they can produce the output with the same weights? How come these networks are sharing weights?

Also, Let us suppose it is just one network (unable to assume, how, please help), then at the first epoch, it'll give outputs with default weights (if used ImageNet). But when the gradients flow back to the network, how are these updated? Because There are 3 different paths going from the same model and how the gradients will flow back to theese paths? Parallelly is not possible (I can't think how) and if sequentially, how that one either because outputs were provided sequentially but gradients can't flow back that way?

• it seems you use some python package, please mention it in your question in case it is language-specific – Nikos M. Jan 4 at 19:01
• @NikosM. No! No! I can ask that thing at Stackoverflow but what I want to ask here is that How the Weights are Updated ( Gradients Flow back) into the network in a Siamese Network? – Deshwal Jan 5 at 5:28

## 1 Answer

You only create ONE model for a siamese network that you pass your inputs to.

(You have created three models in the example)

So in your triplet case you would pass the three inputs seperatly to the network and compute a loss and backprop it.

The gradients are then updated just like training any neural net.

• Yes Exactly, But what about the updation process? I mean, 3 branches are given Sequentially so how'd you update it Parallely? – Deshwal Jan 5 at 8:58
• Three forward passes to create the 3 representations so you can calculate a triplet loss, then one backward pass for the updation process. – Isbister Jan 5 at 9:04
• Perhaps a performance boost would be to make one forward pass with the 3 samples in a batch? – Isbister Jan 5 at 9:06
• so you mean to say that Loss created will be for all the 3 branches and the gradients will be shared among the 3 and will be updated one by one by each Branch? – Deshwal Jan 5 at 10:03
• Not entirely sure I follow what you mean with branches. Just pass the 3 inputs in, calculate triplet loss, and backprop it once. Think of it as a normal classification task, where you forward pass once and backward pass once. But here you will forward pass three times but still only backward pass once. The way of drawing siamese networks with several branches is just for visualization purposes :) – Isbister Jan 5 at 11:12