2
$\begingroup$

I am training multiple neural networks with various parameters. I am trying to average their predictions, but I am not really sure what that means, I am confused about what to average exactly. Here is what I mean: For a single observation in binary classification for example, the final node will give p a value between 0 and 1 (or -1 and 1 if you're using hyperbolic tangent Activation Function), then this p will be rounded to 1 or 0 if it's > 0.5, depending on your decision boundary.

Now, here is what I don't understand, should average p1, p2 and p3 produced by the models before rounding, or I should round the values to True/False responses and then compute the average? and how does that work exactly ?

$\endgroup$
  • $\begingroup$ Can you specify the overview of the problem you are trying to solve... Ie. Binary classification, multi class classification, multi label classification Looks you are trying for Ensembling of various networks. $\endgroup$ – rajeshkumargp Oct 7 '19 at 17:52
1
$\begingroup$

There are multiple ways you could do. All of them are categorized under Ensemble methods in machine learning.

Voting classifiers: which is the simplest way. You just take votes based on the label from all models and uses the majority label. That means, you should first round up all labels to 0 or 1 and then use the majority.

Weighted voting classifiers: similar to previous one, but some models have higher weights in voting.

For more information look here.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ So what happens if I don't round up the values and used some majority function? I am just interested on the reasoning behind it if that's possible. $\endgroup$ – U. User Oct 7 '19 at 17:56
  • $\begingroup$ The reason is, if you have 100 models all are making a prediction between 0 and 1. In some cases, they might behave randomly and if you computes the average, the average may always converge to 0.5. $\endgroup$ – aminrd Oct 7 '19 at 17:59
  • $\begingroup$ God I've been working on this project for months now and nothing made sense really because I wasn't rounding :/ $\endgroup$ – U. User Oct 7 '19 at 18:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.