1
$\begingroup$

TL;DR: Intuition behind the gradient flow in Siamese Network? How can 3 models share the same weights? And if 1 model is used, how Gradients are updated from 3 different paths?

I am trying to build a Siamese Network and as far as I can know, if I have to build a Triplet Loss based Siamese, I have to use 3 different networks. So for simplicity, let us say that my architecture is something like: Please correct the architecture if wrong

    I1 = Input(shape=image_shape)
    I2 = Input(shape=image_shape)
    I3 = Input(shape=image_shape)

    res_m_1 = ResNet50(include_top=False, weights='imagenet', input_tensor=I1, pooling='avg')
    res_m_2 = ResNet50(include_top=False, weights='imagenet', input_tensor=I2, pooling='avg')
    res_m_3 = ResNet50(include_top=False, weights='imagenet', input_tensor=I3, pooling='avg')

    x1 = res_m_1.output
    x2 = res_m_2.output
    x3 = res_m_3.output

    # x = Flatten()(x) or use this one if not using any pooling layer

    ##### ------- ---------------------------- --------- ########

    'NEED HELP AFTER THIS ONE; HOW TO BUILD ARCHITECTURE'
    
    ########### ------------------------------------ ###########

    siamese_model = Model(inputs=[I1,I2], outputs=final_output)
    
 
    siamese_model.compile(loss=some_triplet_loss,optimizer=Adam(),metrics['acc'])

    siamese_model.fit_generator(train_gen,steps_per_epoch=1000,epochs=10,validation_data=validation_data)

My Understanding and Question:: If there are 3 networks in the architecture, how come they can produce the output with the same weights? How come these networks are sharing weights?

Also, let us suppose it is just one network (unable to assume, how, please help), then at the first epoch, it'll give outputs with default weights (if used ImageNet). But when the gradients flow back to the network, how are these updated? Because there are 3 different paths going from the same model and how the gradients will flow back to these paths? Parallelly is not possible (I can't think how) and if sequentially, how that one either because outputs were provided sequentially but gradients can't flow back that way?

Improve this question
$\endgroup$
2
  • $\begingroup$ it seems you use some python package, please mention it in your question in case it is language-specific $\endgroup$
    – Nikos M.
    Jan 4, 2021 at 19:01
  • $\begingroup$ @NikosM. No! No! I can ask that thing at Stackoverflow but what I want to ask here is that How the Weights are Updated ( Gradients Flow back) into the network in a Siamese Network? $\endgroup$
    – Deshwal
    Jan 5, 2021 at 5:28

2 Answers 2

Reset to default
0
$\begingroup$

You only create ONE model for a siamese network that you pass your inputs to.

(You have created three models in the example)

So in your triplet case you would pass the three inputs seperatly to the network and compute a loss and backprop it.

The gradients are then updated just like training any neural net.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Yes Exactly, But what about the updation process? I mean, 3 branches are given Sequentially so how'd you update it Parallely? $\endgroup$
    – Deshwal
    Jan 5, 2021 at 8:58
  • $\begingroup$ Three forward passes to create the 3 representations so you can calculate a triplet loss, then one backward pass for the updation process. $\endgroup$
    – Isbister
    Jan 5, 2021 at 9:04
  • $\begingroup$ Perhaps a performance boost would be to make one forward pass with the 3 samples in a batch? $\endgroup$
    – Isbister
    Jan 5, 2021 at 9:06
  • $\begingroup$ so you mean to say that Loss created will be for all the 3 branches and the gradients will be shared among the 3 and will be updated one by one by each Branch? $\endgroup$
    – Deshwal
    Jan 5, 2021 at 10:03
  • $\begingroup$ Not entirely sure I follow what you mean with branches. Just pass the 3 inputs in, calculate triplet loss, and backprop it once. Think of it as a normal classification task, where you forward pass once and backward pass once. But here you will forward pass three times but still only backward pass once. The way of drawing siamese networks with several branches is just for visualization purposes :) $\endgroup$
    – Isbister
    Jan 5, 2021 at 11:12
0
$\begingroup$

You do three forward passes for the three inputs and calculate one loss. So some modules (maybe all) are used three times. As the gradients depend on the inputs, three gradients get calculated and accumulated!

Source

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.