I was looking into the multi-label classification on the output layer of a Neural Network.
I have 5 Output Neurons where each Neuron can be 1, 0, or -1. independent of other Neurons.
So for example an Output would look like :
I used to take the tanh- activation function and partition the neuron into 3 ( y<-0.5, -0.5<y<0.5, y>0.5) to decide the class in each of those neurons after the prediction. See Figure below.
Are there any better alternatives on how to approach this ternary Output activation?
I stumbled upon this blog post regarding n-ary activation functions which I found very interesting.
I think the newly suggested would make the partitioning mentioned above much more meaningful!
Or do you think I should one-hot-encode my whole system such that a neuron can only have a value of 0 or 1 and then just use sigmoids activation with a threshold of 0.5?