I want to know the effect of Add and Multiply in keras by functionality. The dumb way of thinking is that they are meant to add and multiply keras tensors. I want to know when are they to be used. For example, look at the code below from here. Why use Multiply in spatial_squeeze_excite_block and why use Add in channel_spatial_squeeze_excite? Can we switch Add and Multiply in these functions? Why not?
def spatial_squeeze_excite_block(input):
''' Create a spatial squeeze-excite block
Args:
input: input tensor
Returns: a keras tensor
References
- [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://arxiv.org/abs/1803.02579)
'''
se = Conv2D(1, (1, 1), activation='sigmoid', use_bias=False,
kernel_initializer='he_normal')(input)
x = Multiply([input, se])
return x
def channel_spatial_squeeze_excite(input, ratio=16):
''' Create a spatial squeeze-excite block
Args:
input: input tensor
filters: number of output filters
Returns: a keras tensor
References
- [Squeeze and Excitation Networks](https://arxiv.org/abs/1709.01507)
- [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://arxiv.org/abs/1803.02579)
'''
cse = squeeze_excite_block(input, ratio)
sse = spatial_squeeze_excite_block(input)
x = Add([cse, sse])
return x
