1
$\begingroup$

Thanks in advance for reading my question!

I've been using CNNs to classify text using Keras and TF. My data is strings "I read the news" or "I read machine learning news" and my labels are tags: Data Science, Reporter, Child...

My issues is that each text can have multiple labels attached to it. How should I construct my target such that I can capture each of those multiple outputs?

Description, Tag "I read the news", Child "I read the news", Reporter "I read machine learning news", Data Science "I read machine learning news", Reporter

Improve this question
$\endgroup$
-1
$\begingroup$

CNN's (generally) use softmax as activation function at the last layer, which gives a probability distribution over all possible labels. You use some kind of loss function to optimize the CNN to generate "right" label for a novel sample.

Loss function used in such case is cross-entropy loss or KL-divergence which measures how close two distributions are. For multiclass classification (different from your case, where each input example can have only one label), desired distribution is (0, 0, .... , 1, ..., 0) where 1 appears at the position of actual label. Loss function penalizes the distribution which is "far" from this distribution.

For your case you can encode your output as (0, 0, 1, 0, ... 1, ..,) so on where you put ones at positions of the labels which are associated with your input and use KL-divergence as loss function and optimize your CNN to minimize the loss.

Improve this answer
$\endgroup$
  • $\begingroup$ CNNs are not constrained to only use softmax in output layer. A simple logistic function on output layer would work much better for OP. $\endgroup$ – Neil Slater Sep 28 '16 at 19:39
  • $\begingroup$ Logistic function is a special case of softmax with two classes $\endgroup$ – SHASHANK GUPTA Sep 28 '16 at 19:44
  • 3
    $\begingroup$ No it is not. A single logistic function output using log loss can be equivalent to two-output softmax using multiclass logloss (in terms of end result from the network). But the OP will want multiple outputs with logistic function applied, which is not the same as softmax with any number of outputs. Most importantly, softmax estimates exclusive class probabilities, and OP needs non-exlusive probabilities, which can be modelled with an output layer of logistic neurons, but not with a softmax layer. $\endgroup$ – Neil Slater Sep 28 '16 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.