-1
$\begingroup$

I am creating a binary classifier in Keras and here;s the code

model = Sequential()
    model.add(Dense(30, input_dim=60, activation='relu'))
    model.add(Dense(1, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

I was wondering, if the dataset's range is changed from [0,1] to [-1,1], shouldn't I be able to use tanh as the activation of the output layer? If so, what are the advantages of one activation over the other?

$\endgroup$
0
$\begingroup$

For a binary classifier, it is prominent to use sigmoid as the activation function. The sigmoid function's range is $[ 0 , 1 ]$. That makes sense since we need a probability which could determine two ( binary ) classes i.e 0 and 1.

If you are using tanh ( hyperbolic tangent ) it will produce an output which ranges from -1 to 1. In this case, we cannot determine the binary classes.

Hence, we require sigmoid rather than tanh especially for binary classification.

$\endgroup$
  • 2
    $\begingroup$ Worth adding that the OP's chosen loss function binary_croseentroyp, also will not function correctly for input range $[-1,1]$ $\endgroup$ – Neil Slater Aug 18 '19 at 6:46
0
$\begingroup$

A problem where you classify an example as belonging to one of two classes.

The problem is framed as predicting the likelihood of an example belonging to class one, e.g. the class that you assign the integer value 1, whereas the other class is assigned the value 0.

Output Layer Configuration: One node with a sigmoid activation unit. Loss Function: Binary-Cross-Entropy, also referred to as Logarithmic loss. Another point to note is softmax is a generalization of sigmoid for producing probabilities for multi-class problems so that the probabilities strictly sum to 0,hence rather than using tanh go for sigmoid or either softmax(it is same as sigmoid for binary classification problems).

$\endgroup$
0
$\begingroup$

If you label your data using -1 and 1 as classes, then yes you can. However, there are two reasons why data scientists normally prefer Sigmoid activations:

  • Loss functions, such as cross entropy based, are designed for data in the [0, 1] interval.

  • Better interpretability: data in [0, 1] can be thought as probabilities of belonging to acertain class, or as a model's confidence about it.

But yeah, you can use Tanh and train useful models with it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.