2
$\begingroup$

I'm using keras in R to predict financial time series. I guess that ANN converges. enter image description here

But when I do predictions I can see that probabilities don't sum up to 1

enter image description here

I'm using relu as final activation and loss = "binary_crossentropy". Does that mean the net doesn't converge or do I miss something else?

$\endgroup$

2 Answers 2

1
$\begingroup$

Even if input to a neural netwrk are scaled or normalised, the raw output values can still go outside of that range.

In your case, the output values are being interpreted as to make a binary YES/NO decision, but the raw values cannot necessarily be interpreted as raw probabilities! They are merely the final activations of the network.

To get what you expect, the final activations are usually passed through a softmax function, which essentially squashes the values you see in your table to sum to 1 on each row - this allows us to treat them as probabilities to make the final classification.

In practice, this means simply adding the softmax activation to your final Dense layer in Keras (activation="softmax") and then compile the model using: loss="categorical_crossentropy"

$\endgroup$
2
  • $\begingroup$ I'm using relu as final activation and loss = "binary_crossentropy" $\endgroup$
    – Andrew
    Jan 13, 2019 at 21:23
  • 2
    $\begingroup$ It is general advised not to use a non-linear activation in the final output, as this adds a step which the last layer doesn't directly decide - or rather it has to really learn how a ReLU works. It is unnecessary. If you want that layer there, just add another one on top with the sigmoid activation. $\endgroup$
    – n1k31t4
    Jan 13, 2019 at 21:34
0
$\begingroup$

As you have 2 numbers, your network has two output nodes. For example, Female and Male. In binary classification the output nodes are independent and the prediction for each node is from 0 to 1. So, you should consider a threshold (usually 0.5). Then if the prediction value is upper than this threshold for Male, you consider the image as Male.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.