10
$\begingroup$

I have a corpus of text with a corresponding topics. For example "A rapper Tupac was shot in LA" and it was labelled as ["celebrity", "murder"]. So basically each vector of features can have many labels (not the same amount. The first feature vector can have 3 labels, second 1, third 5).

If I would have just one label corresponded to each text, I would try a Naive Bayes classifier, but I do not really know how should I proceed if I can have many labels.

Is there any way to transform Naive Bayes into multi label classification problem (if there is a better approach - please let me know)?

P.S. few things about the data I have.

  • approximately 10.000 elements in the dataset
  • text is approximately 2-3 sentences
  • maximum 7 labels per text
Improve this question
$\endgroup$
2
  • $\begingroup$ @fordprefect Multinomial Naive Bayes uses a multinomial distribution for the probabilities of some feature given a class: $p(f_i|c)$. The OP wants a classifier to manage multiple outputs as TheGrimmScientist described. $\endgroup$
    – r_31415
    Commented Dec 12, 2014 at 1:03
  • $\begingroup$ en.wikipedia.org/wiki/Multi-label_classification $\endgroup$
    – D.W.
    Commented Jan 4, 2017 at 21:33

1 Answer 1

Reset to default
6
$\begingroup$

For starters, Naive Bayes is probably not appropriate here. It assumes independence among the inputs (hence the "Naive") and words in a sentence are very dependent.

But, assuming you really want to run with NB as an algorithm to start your experimentation, there are two options I'd consider:

Ungraceful: Lots of NB classifiers

This would be an alternative approach. Make a corupus of all the words observed as your vector of inputs. Make a corpus off all the tags that are observed as your vector of outputs. An NB classifier with multiple outputs is the equivalent of having multiple NB classifiers with one output each (so do whichever is easier to implement in whatever software framework you're using). Treat each element as a training sample where a given input (a word) is a 1 if that word is present and a 0 if that word isn't. Use the same binary scheme for the output.

This brute forces the application of the NB Classifier to your data, and leaves you to find meaning by still haivng to mine the huge set of classifiers you'll be left with.

More Graceful: Process your data

This is the approach I'd recommend if you want to run with one multiple-class NB Classifier.

Your goal here is to figure out how to map each set of tags to a single class. I'm sure there is some sort of clustering scheme or network analysis (perhaps ["celebrity"] linked to ["murder"] could become a the segment ["debauchery"]) that will sensibly map your tags to one single cluster. If you treat tags as nodes and two given tags together as links, then you'll want to look into community detection algorithms (which is where I'd start). But, if you just want something working, then some sort of hack on the tags that converts a list of tags to only the tag that's most commonly seen in your dataset would be enough.

This method front-loads the work of cleaning your data and would make the NB Classifier's output easier to understand.

Improve this answer
$\endgroup$
3
  • $\begingroup$ thank you for your input, if you have an idea what would be better then NB, please let me know. $\endgroup$
    – Salvador Dali
    Commented Dec 12, 2014 at 0:17
  • 1
    $\begingroup$ the "existing solutions" section of this slide deck holds everything I would know to reply with, plus more (assuming you're not needing specifically a classifier and just want a way to use tags). I hope it's useful to you. $\endgroup$
    – TheGrimmScientist
    Commented Dec 12, 2014 at 1:48
  • $\begingroup$ @TheGrimmScientist Wouldn't it be reasonable to use the first approach? If you have a vector of features f1, f2, f3 and let's say, 3 labels for this vector, we can partition that into 3 vectors (all containing the same features f1, f2, f3) with different labels as outputs. Then it is possible to use Naive Bayes as usual. I'm not sure if that is what you had in mind. $\endgroup$
    – r_31415
    Commented Dec 12, 2014 at 2:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.