2
$\begingroup$

I'm doing a deep learning model using tensorflow and keras. I have a question about the output architecture. I want to classify between two classes, images with defects and images without defects, I've built a CNN based on VGG16 but smaller. The problem is basically a binary classification.It's possible to use one unique neuron on the output using the sigmoid activation and the binary crossentropy function as loss. The other option is to use two neurons, using softmax and categorical crossentropy as loss (of course, using a onehot ecoder to represent the 2 labels). I've been looking for a detailed explanation of the difference between this two options but I have not found response of what I really want to know.

This is a related question I've read: https://stats.stackexchange.com/a/260537/295456 Here is said that binary crossentropy is just a special case of categorical crossentropy.

Is there any advantage in any of this approach? I'd like to know the advantages and disadvantages of both options. The network would converge faster/slower? There would be more parameters to train? One of this options can be considered as bad desing?

This is just a schematic code to represent the two aproches:

## Opt 1.
targets = [[0,1], [1,0],...]
...
model.add(Dense(units = 2, activation = 'softmax'))
model.compile(loss = 'categorical_crossentropy')

## Opt 2.
targets = [1,0,1,...]
...
model.add(Dense(units = 1, activation = 'sigmoid'))
model.compile(loss = 'binary_crossentropy')
$\endgroup$

1 Answer 1

0
$\begingroup$

In the case of binary classification, those two methods are equivalent. There will be no meaningful difference in training between those two methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.