0
$\begingroup$

I currently have a multi-label classification problem, for which I am using keras to build a neural network as follows:

n_cols = dataset.shape[1]
print(n_cols)

model = Sequential()
model.add(Dense(128, activation='relu', input_shape=(n_cols,)))
model.add(Dense(64, activation='relu'))
model.add(Dense(26, activation='sigmoid')) # Sigmoid for multi-label classification

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.5, nesterov=True)
#RMSprop
model.compile(loss='binary_crossentropy', optimizer='RMSprop', metrics=['accuracy'])

model.summary()

## Fit the model ##
early_stopping_monitor = EarlyStopping(patience=20)
history = model.fit(dataset, labels, validation_split=0.33, epochs=30, callbacks=[early_stopping_monitor])

plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Test'], loc='upper left')
plt.show()

I was informed that for multi-label classification, we use binary_crossentropy for the loss while having sigmoid for activation in the final layer (output layer). However, with this I am getting a resulting accuracy and val_accuracy of ~0.0931 and ~0.0937 respectively.

For the multi-label classification, is using the accuracy metric the best fit? I've looked around and some suggest that other metrics such as binary_accuracy may be better..

So the question is, how can one best evaluate the multi-label classification?

EDIT: For reference, I have 26 label columns in my target "classes" and the dataset consist of 21 columns. The entire dataset the model is trained on has ~82k samples.

$\endgroup$
0
$\begingroup$

Binary crossentropy is for two class problem. You must use sparse categorical crossentropy for your multiclass classification problem and softmax in the last layer not sigmoid. Sigmoid and binary crissentropy is for two class problem while sparse categorical crossentropy and softmax is for multiclass problem.

| improve this answer | |
$\endgroup$
  • $\begingroup$ This is not a multi-class problem though, this is a multi-label problem, where a sample can be part of more than one class. $\endgroup$ – rshah Jul 16 at 21:46
  • $\begingroup$ Then you should not use sigmoid or softmax at all. If the number of label is fixed for every sample you can use different head for different sets of label $\endgroup$ – SrJ Jul 16 at 21:52
  • $\begingroup$ can you elaborate? $\endgroup$ – rshah Jul 17 at 9:31
0
$\begingroup$

you are correct with using sigmoid+binary CE for multi-label classification problem. On the other hand, try to think about how accuracy would be defined for this problem (https://keras.io/api/metrics/accuracy_metrics/#accuracy-class)? I would use categorical accuracy!

Maks

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.