# Comparison between addition and multiplication function in deep neural network?

I designed a specific Convolution Neural Network to study in the area of image processing. The network has a part that there are two tensors which have to be transformed into a tensor in order to be fed to the next layer. This situation happen in several points of the network. In fact, there are several operations such as addition, multiplication, etc. The results of the network are a bit better when I use the addition pyramid pooling module (the second image between two convolution) and multiply function (in last step of the network). I used tf.math.add and tf.math.multiply which perform the operation element-wisely. The whole network is shown in the first image. The second image represents the pyramid pooling module which includes several scale images. I am looking forward to the addition and multiplication function's attribute in a deep neural network.

The question is:

Why does the addition function (between conv1 and conv2) indicate better final performance in Accuracy (precision) and mean Intersection of Union(mIoU) compared to multiplication and concatenation when I unify two tensors into one tensor?

• Please ask only one question per post. Also, i'ts unclear what you mean by "the most important features of addition vs multiplication", and it's not clear how you are using addition or multiplication, so I don't think any of these questions are answerable in their current form. If you can edit your question to address this feedback, I encourage you to do so. – D.W. Feb 16 '19 at 21:46
• I don't quite understand this, but typically you're using dense layers for non-linear transformations. If all it does is sum combinations of inputs, it's a linear transformation. That almost surely defeats the purpose of what you're using it for. – Sean Owen Feb 17 '19 at 0:56
• dear @SeanOwen, I explained that in a specific part of the network there are two tensors which have to unify in order to feed into the next layer, in this case, there are several choices. one of these choices is basic mathematics operation such as addition, multiplication. we performed several experiments with each of these operations. I change the question and make it narrow. could u look at it again? – amir Maleki Feb 17 '19 at 8:14
• Dear @D.W., I change the question, I used basic addition which is provided by Tensorflow, tf.math.add which returns the a+b element-wise (each of a and b is equal tensors) and for multiplication I also used the tf.math.multiply function. – amir Maleki Feb 17 '19 at 8:26
• What do you mean 'unify'? there is no general answer to this. Which operation you use depends on what you are trying to do, and, practically, which one works better. If you mean to add things, you add them. – Sean Owen Feb 17 '19 at 15:36