# Activation and Loss Function not chosen correctly when use Neural Network

I have three classes for my text dataset before.

These are my classes:

0 = Cat
1 = Not Both
2 = Dog

Then I use this code:

df_result = df[df["class"] != 1]


So, now my classes are 0 and 2. When I use neural networks, what can I choose for the Loss and Activation? Then, what should I do for choose the last Dense of neural network's model?

Before, my code like this:

model.add(Dense(2, activation = 'softmax'))
history = model.compile(loss = sparse_categorical_crossentropy,


I do not know if it makes a bad accuracy and bad validation accuracy or not. So, I am still confused. Please, give me a hand.

There are 2 possible scenarios here.

You can use all 3 categories and build a multiclass classification model, where the output layer has 3 neurons, activation function is softmax and loss is sparse cross entropy loss. If you choose to go with this method then make sure to use LabelEncoder to encode your target variable.

The other scenario is that you use only 2 classes as you suggest. In that case you can build a binary classification model where the output layer has 1 neuron, activation function is sigmoid and loss is binary cross entropy loss. If you choose to go with this method then make sure you use OneHotEncoder to encode your target variable.

I would suggest try both methods and see which one gives better results.

Cheers!

• Hi. Thanks before for the explanation. But, when I use OneHotEncoder for two classes (means: "0" for Cat and "2" for Dog, because I drop class "1") should I change the names of classes? Like class "2", I gonna change it to class "1" because I already drop it and now class "1" is available. When I trying that, am I right? Dec 9, 2021 at 8:18
• After you drop cone of the class, you have classes Cat and Dog remaining. So when you OneHot Encode them, python will automatically assign 0 for Cat and 1 for Dog. you don't need to do it manually. Dec 9, 2021 at 16:14
• Thank you for your advice. It's gonna be help me now Dec 10, 2021 at 5:16

If you are performing classification using just two classes you have two options: (1) having a single output for your model to predict with a value of 0/1 indicating if the sample belongs to one of the classes, or (2) use both labels (either use one-hot encoding or using the labels as is) and have the model predict both the probability that the sample belongs to the first and second class. In the first case you would generally use a sigmoid activation with a binary cross entropy loss function, in the second case you would generally use a softmax activation with a (sparse) cross entropy loss function.

• Hi. Thank you for the explanation. I wish this suggestions make help for me. I'll try it Dec 9, 2021 at 8:12