2
$\begingroup$

I have a set of documents (1 to 11) for which the labeling is done.

Lets Assume:

Doc No: 1,3,5,7 - Belongs to Type A
Doc No: 2,4,9 - Belongs to Type B
Doc No: 8,10 - Belongs to Type C
Doc No, 6,11 - Belongs to No one

Now, Lets say i have new incoming docs - 11,12,13 .. and so on, and i want to know which Type (A, B, C or none ) they belongs to based on text similarity of existing docs in that Type . Can someone please suggest how to achieve this ?

Should i create my own corpus of data and consider it a supervised problem.

Need guidance and support here. Thanks

$\endgroup$
0
$\begingroup$

Yes this is a supervised problem. I'd suggest following the example in this article.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I would consider some unsupervised techniques followed by supervised labelling. Basically, represent your incoming documents as dense vectors and compute similarity between the already labelled documents. Then, label them with the most similar document.

Ideas on how to solve:

  • Run Latent Dirichlet Allocation (LDA) on all the documents.
  • Each labelled document is then the probability distribution over topics
  • It looks like Document 1 : [0.1 0.3 0.0 ...], Document 2: [0.8 0.3 0.1 ...], ...
  • Finally, for all incoming documents, compute similarity with all the already labelled documents.
  • Label the incoming document with the label of most similar document, which is already labelled.

Another idea:

  • Replace the LDA with Word2Vec based models.
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.