The sum of multi-class prediction is not 1 using tensorflow and keras?

Question

I am studying how to do text classification with multiple labels using tensorflow. Let's say my model is like:

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, 50, weights=[embedding_matrix], trainable=False),
    tf.keras.layers.LSTM(128),
    tf.keras.layers.Dense(4, activation='sigmoid')])
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=tf.metrics.categorical_accuracy)

I have 4 classes, the prediction function has the following results:

pred=model.predict(X_test)
pred
array([[0.915674  , 0.4272042 , 0.69613266, 0.3520468 ],
       [0.915674  , 0.42720422, 0.69613266, 0.35204676],
       [0.915674  , 0.4272042 , 0.69613266, 0.3520468 ],
       [0.9156739 , 0.42720422, 0.69613266, 0.3520468 ],
   ......

You can see that every data has 4 prediction values. But the sum of them is not 1, which I do not understand. Did I do something wrong, or how to interpret this?

Answer 1

To get the summation in the last layer to be one, you need to use Softmax activation not Sigmoid. Often Sigmoid is used in a special case when there is a binary classification problem.