I have 3 neural networks where each has as a different activation function: Sigmoid, Tanh and Softmax. I am planning to average their final predictions, but as we know the functions doesn't have the same range values.
P = (P1 + P2 + P3)/3
0 < P1 < 1, -1 < P2 < 1, 0 < P3 < 1
Can I directly average the predictions or I need to perform a normalization to have all prediction fall into the same interval ?