I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is what I mean: For a single observation in binary classification for example, the final node will give p a value between 0 and 1 (or -1 and 1 if you're using hyperbolic tangent Activation Function), then this p will be rounded to 1 or 0 if it's > 0.5, depending on your decision boundary.

Now, here is what I don't understand, should average p1, p2 and p3 produced by the models before rounding, or I should round the values to True/False responses and then compute the average? and how does that work exactly ?

  • $\begingroup$ Can you specify the overview of the problem you are trying to solve... Ie. Binary classification, multi class classification, multi label classification Looks you are trying for Ensembling of various networks. $\endgroup$ Oct 7, 2019 at 17:52

1 Answer 1


There are multiple ways you could do. All of them are categorized under Ensemble methods in machine learning.

Voting classifiers: which is the simplest way. You just take votes based on the label from all models and uses the majority label. That means, you should first round up all labels to 0 or 1 and then use the majority.

Weighted voting classifiers: similar to previous one, but some models have higher weights in voting.

For more information look here.

  • $\begingroup$ So what happens if I don't round up the values and used some majority function? I am just interested on the reasoning behind it if that's possible. $\endgroup$
    – U. User
    Oct 7, 2019 at 17:56
  • $\begingroup$ The reason is, if you have 100 models all are making a prediction between 0 and 1. In some cases, they might behave randomly and if you computes the average, the average may always converge to 0.5. $\endgroup$
    – aminrd
    Oct 7, 2019 at 17:59
  • $\begingroup$ God I've been working on this project for months now and nothing made sense really because I wasn't rounding :/ $\endgroup$
    – U. User
    Oct 7, 2019 at 18:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.