2
$\begingroup$

I just wrote an MLP in Python. After having trained it, I pass in some test data to see the result, and I get an array of decimal numbers at the output, rather than the desired binary output.

For example, I'll get [0.43, 0.56, 0.1, 0.8], rather than [0, 1, 0,1].

What can I do to achieve this?

Thanks

Liam

Improve this question
$\endgroup$
4
  • $\begingroup$ @nbro 3 hidden layers, each with 50 neurons, each using a sigmoid activation function. 9 output neurons (recognizing digits) and 1500 input neurons (for MFCC sound files) $\endgroup$
    – Liam F-A
    Commented Feb 15, 2019 at 1:49
  • $\begingroup$ @nbro, Ok, that seems like it would work, but it seems like softmax returns a probability, and the program would take the highest one. But what would I do if I want to encode my output with less outputs neurons than the amount of possible results. For example, representing an 9 as [1,0,0,1], as opposed to [0,0,0,0,0,0,0,0,1] $\endgroup$
    – Liam F-A
    Commented Feb 15, 2019 at 2:17
  • $\begingroup$ You can do that, with sigmoid, but then you will force your model to learn binary encoding which is something that you don't really want it to waste its' computations on. So it is not really adviced. $\endgroup$
    – Simon Larsson
    Commented Feb 15, 2019 at 9:05
  • $\begingroup$ @nbro, ok so I tried the softmax which actually seemed to improve things, but only to convert the output of the NN. Should I also be using softmax as the activation function during the training phase ? If so, how do I compute its derivative (for backpropagation) ? $\endgroup$
    – Liam F-A
    Commented Feb 16, 2019 at 3:57

1 Answer 1

Reset to default
2
$\begingroup$

This looks like a case of the model outputting the probability of being in category 1. It then is up to you to decide on the cutoff.

You give an example of an output of $(0.43, 0.56, 0.1, 0.8)$. If your cutoff is $0.5$, you’d get classifications of $(0, 1, 0, 1)$. If you set your cutoff at $0.2$, which you’re allowed to do, you get classifications of $(1,1,1,0)$. There should be a way to do this in whatever software you’re using. If all else fails, you can loop through your vector of probability values and produce a new vector with some conditional statements about if the value exceeds the threshold and if the value does not. I’ll give some pseudo-code.

for value...
  if value>threshold
    new value =1
  else
    new value =0

This leads to ROC curves and area under the curve, where the model has its accuracy assessed at all cutoffs. Perhaps even better is to use a strictly proper scoring rule. You can read more about this on the Cross Validated Stack, stats.stackexchange.com. Look out for posts on this topic by Frank Harrell. Shamelessly, I will mention a post of mine where I somewhat challenge this idea: https://stats.stackexchange.com/questions/464636/proper-scoring-rule-when-there-is-a-decision-to-make-e-g-spam-vs-ham-email.

Loosely speaking, a strictly proper scoring rule penalizes a model for lacking confidence in its prediction. Outputting a probability of being in class 1 of $0.56$ is not as good of a prediction as $0.99$. Think of the first as, “Sure, I guess it’s category 1,” while the second is, “Oh heck yes it’s category 1!”

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.