Activation and Loss Function not chosen correctly when use Neural Network

Question

I have three classes for my text dataset before.

These are my classes:

0 = Cat
1 = Not Both
2 = Dog

Then I use this code:

df_result = df[df["class"] != 1]

So, now my classes are 0 and 2. When I use neural networks, what can I choose for the Loss and Activation? Then, what should I do for choose the last Dense of neural network's model?

Before, my code like this:

model.add(Dense(2, activation = 'softmax'))
history = model.compile(loss = sparse_categorical_crossentropy,    
optimizer = 'adam', metrics=['accuracy'])

I do not know if it makes a bad accuracy and bad validation accuracy or not. So, I am still confused. Please, give me a hand.

Answer 1

There are 2 possible scenarios here.

You can use all 3 categories and build a multiclass classification model, where the output layer has 3 neurons, activation function is softmax and loss is sparse cross entropy loss. If you choose to go with this method then make sure to use LabelEncoder to encode your target variable.

The other scenario is that you use only 2 classes as you suggest. In that case you can build a binary classification model where the output layer has 1 neuron, activation function is sigmoid and loss is binary cross entropy loss. If you choose to go with this method then make sure you use OneHotEncoder to encode your target variable.

I would suggest try both methods and see which one gives better results.

Cheers!

Answer 2

If you are performing classification using just two classes you have two options: (1) having a single output for your model to predict with a value of 0/1 indicating if the sample belongs to one of the classes, or (2) use both labels (either use one-hot encoding or using the labels as is) and have the model predict both the probability that the sample belongs to the first and second class. In the first case you would generally use a sigmoid activation with a binary cross entropy loss function, in the second case you would generally use a softmax activation with a (sparse) cross entropy loss function.