2
$\begingroup$

Suppose I have a piece of writing and I want to assign probabilities to different genres (classes) based on its contents. For example

Text #1 : Comedy 10%, Drama 50%, Fiction 20%, Romance 1%, Mythology 5%, Adventure 10%

Text #2 : Comedy 40%, Drama 3%, Fiction 2%, Romance 30%, Mythology 5%, Adventure 10%

We have given keywords possibly ngrams in each class through which we make a comparison

Class 1 Comedy : k11, k12, ..., k1m

Class 2 Drama : k21, k22, ..., k2n

Class 3 Fiction : k31, k32, ..., k3o

Class 4 Romance : k41, k42, ..., k4p

Class 5 Mythology : k51, k52, ..., k5q

Class 6 Adventure: k61, k62, ..., k6r

What can be the best probabilistic model that we can use for this task

Improve this question
$\endgroup$
7
  • $\begingroup$ Is your list of genre keyword associations the only training data you have? Do you have a set of texts that are labelled with the correct genres? $\endgroup$
    – Imran
    Commented Jan 11, 2018 at 14:30
  • $\begingroup$ @Imran No Data is pretty less here, we only have a list of n-gram keywords per genre. More keyword match should result in higher probability and less keyword should result in lesser probability per genre. $\endgroup$
    – Atinesh
    Commented Jan 11, 2018 at 17:08
  • $\begingroup$ Can a text have multiple genres? In other words, is it OK if the assigned probabilities add up to more than 1? $\endgroup$
    – Imran
    Commented Jan 11, 2018 at 17:10
  • $\begingroup$ @Imran I can easily use Naive Bayes here by considering each genre as a different class but it will distribute the probability across multiple classes. What I want is to assign probability individually i.e. per class. $\endgroup$
    – Atinesh
    Commented Jan 11, 2018 at 17:13
  • $\begingroup$ So this would be a valid prediction: Comedy 90%, Drama 10%, Fiction 70% ... ? $\endgroup$
    – Imran
    Commented Jan 11, 2018 at 17:22

1 Answer 1

Reset to default
1
$\begingroup$

If I understand correctly, we are interested in soft multilabel classification, where a single text can have multiple correct genres.

According to your comment, we don't have any training data, just a list of keywords associated with each genre.

We can try computing the similarity between each document and each keyword list:

  1. Normalize the document (convert to lowercase, remove punctuation, diacritics, non-alphanums, etc)
  2. Remove stopwords
  3. Convert the document to tf-idf vector over our genre keyword vocabulary: Each document gets an n-length vector where each entry is the frequency of the ith genre keyword in the document. Normalize this vector to magnitude 1.
  4. Convert each genre keyword list to a tf-idf vector in the same way (again over the keyword vocabulary for all genres).
  5. Compute the cosine similarity between the document vector and each genre vector.

For each document, this will give us a number in the range [0,1] for each genre. For example:

         Comedy Drama Fiction Romance Mythology Adventure
Text #1: 0.15   0.11  0.03    0.00    0.00      0.07

If we were doing single label classification we could normalize each row to add up to 1 and we might have a working model. However there is no such trick for multilabel classification here. We don't have a good way to calibrate these values into probability estimates.

At this point the only solution I see is to build a small training set so we can fit our model to actual data.

After gathering some training examples, we can run a multilabel regression with sigmoid activation and binary crossentropy loss with the cosine similarities as input features to get a probability estimate for each class.

Using this method our list of genre keywords will at least save us having to build a large training set to solve the problem directly with bag-of-words or similar approaches.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for detailed explanation, I understand what you are trying to say. Can we modify Naive Bayes for multi-label classification. $\endgroup$
    – Atinesh
    Commented Jan 12, 2018 at 5:32
  • $\begingroup$ Could you update your question to describe how you are calculating prior and likelihood for Naive Bayes? Then I can try and help you extend it to multi-label classification $\endgroup$
    – Imran
    Commented Jan 12, 2018 at 6:08
  • $\begingroup$ Training Naive Bayes simply by considering each genre as a different class. I read some articles they mentioned that Naive Bayes won't give better results for this task (i.e. Multilabel Classification). I think I should stick with your approach. $\endgroup$
    – Atinesh
    Commented Jan 12, 2018 at 8:02
  • $\begingroup$ Yes you could train seperate 1-vs-all Naive Bayes classifiers for each class, but you still need a training set. Just having a list of keywords does not tell you about the prior distribution of classes nor the probability of a class given a piece of text. For both of these you need to fit to labelled training data. Naive Bayes isn't magic! $\endgroup$
    – Imran
    Commented Jan 15, 2018 at 6:52
  • $\begingroup$ Hello @Imran Is there a need to down-weight term frequency of keywords. TF-IDF is widely used for text classification but here our task is multi label Classification as you mentioned i.e to assign probabilities to different labels. I believe creating a TF vector by CountVectorizer() would work fine because here we are concerned more with presence or absence of keyword in a document rather than how important is it as compared to other documents. What do you think ? $\endgroup$
    – Atinesh
    Commented Jan 17, 2018 at 17:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.