# When do I use Multiply and Add

I want to know the effect of Add and Multiply in keras by functionality. The dumb way of thinking is that they are meant to add and multiply keras tensors. I want to know when are they to be used. For example, look at the code below from here. Why use Multiply in spatial_squeeze_excite_block and why use Add in channel_spatial_squeeze_excite? Can we switch Add and Multiply in these functions? Why not?

def spatial_squeeze_excite_block(input):
''' Create a spatial squeeze-excite block
Args:
input: input tensor
Returns: a keras tensor
References
-   [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://arxiv.org/abs/1803.02579)
'''

se = Conv2D(1, (1, 1), activation='sigmoid', use_bias=False,
kernel_initializer='he_normal')(input)

x = Multiply([input, se])
return x

def channel_spatial_squeeze_excite(input, ratio=16):
''' Create a spatial squeeze-excite block
Args:
input: input tensor
filters: number of output filters
Returns: a keras tensor
References
-   [Squeeze and Excitation Networks](https://arxiv.org/abs/1709.01507)
-   [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://arxiv.org/abs/1803.02579)
'''

cse = squeeze_excite_block(input, ratio)
sse = spatial_squeeze_excite_block(input)

x = Add([cse, sse])
return x

• About the multply operation : in the spatial_squeeze_excite_block, you compute a feature map that could be interpreted as a mask (values between 0 and 1 because of sigmoid activation), multiplying the mask with the initial tensor will effectively mask some values from the initial tensor. (very much like in a attention mechanism even though there is no attention here) – mprouveur Oct 2 '20 at 11:12
• In general I'd say the Add operation between layers outputs is meant to gather the information of the 2 layers (you could concatenate but here you don't increase the dimension ) – mprouveur Oct 2 '20 at 11:16