# What are the best activation functions for Binary text classification in neural networks?

I know that there are many activation functions like Relu, sigmoid, tanh ..etc, I just want to know the best one for my case - Binary text classification.

I have heard that Relu is best for Binary classification (not sure if im correct)

I have used keras to train a model, which is 2 layer , Dense 512, dropout 0.3, activation = "Relu" for these layers,

But the predictions are not upto the mark.

I have also changed the Dense units to 1024, keeping others same, but still I got bad predictions. (Validation accuracy 50%)

So, can i use other activations, or change my model layers (add few more layers) ??

What can be the best option ?

• Have a look at the "universal approximation theorem" and the "no free lunch" theorem to see that it is difficult to answer your question correctly. Sep 11 '20 at 21:44

Relu often works well, but ultimately you need to try. If you really have only one hidden layer, you might have a model which cannot properly learn.

I guess you use a sequential, dense model architecture? Try to add more hidden layers. Don‘t add too much capacity for a start. You could try something like 256, 8, 8, 1 with relu.

Dropout is good to prevent overfitting. It is not important for learning per se.

Just a remark for future posts: always include your basic model code if possible. So people can understand what you are really up to. Also a little data description is useful. Especially with text, preprocessing is really important. So the way you treated your data before it goes into the model is relevant.

Sigmoid activation produces an output between 0,1 making it suited for binary classification. As far as hyper parameter tuning goes, experimenting is always necessary.

As you are not specific about how to convert text input into numbers to feed to the model, I can only answer your question in general.

Firstly, for the last layer of binary classification, the activation function is normally softmax (if you define the last layer with 2 nodes) or sigmoid (if the last layer has 1 node).

For other layers, it is hard to tell sigmoid or relu is better. But in my experience, relu works better with more complicated models.