1
$\begingroup$

Can anyone suggest a machine learning algorithm that would be useful for identifying the organization a document belongs to? I'm working with a relatively large number of organizations (on the order of 1000s). Naturally, the input data is unstructured, but most documents look almost, but not quite, the same. Also, there are some organizations that follow more then a one template, again, in that "almost but not quite" manner.

Documents are in PDF format, and are reports of some kind and can contain several rectangular regions of text and tables. Most of them are not more than five pages long.

New organizations can eventually appear - how does interfere with algorithm?

Improve this question
$\endgroup$
2
  • $\begingroup$ If you're looking for a search term, the problem is called "online document classification". The biggest challenge in your scenario is that the classes are dynamic. $\endgroup$
    – Emre
    Commented Jul 25, 2015 at 3:07
  • $\begingroup$ "most documents look almost the same" - do you mean regardless of organisation, they look the same? Or each organisation's documents look the same and different to other organisations? Is there any trick like looking for the org's logo you could use? Do you know the possible set of org names? Could you just look for that in the text? Do you have a training data set classified with correct orgs? $\endgroup$
    – Spacedman
    Commented Oct 21, 2015 at 13:12

4 Answers 4

Reset to default
1
$\begingroup$

Additional to the content of the document, the form could contain even more valuable information. I can't comment on the formatting of pdf files, but it should be possible to exploit the pdf metadata. Author, Title, subject or keywords could provide some clue pointing to a common origin in the same organization.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Also, the column layout, presence of delimiter images, font, inter-character/line spacing, text-density. Pre-processing the documents with a document-image understanding framework can provide the basis for extracting these layout features. Cermine is an open-source framework, which among other things, extracts meta-data from PDF's. $\endgroup$
    – levi
    Commented Oct 26, 2015 at 18:15
0
$\begingroup$

Try Decision Trees,Support Vector Machines or Naive-Bayes method with TF-IDF weighting for creating document vectors and check the Precision/Recall/F Measure scores. But this will not deal with unknown/new organizations, they will get classified to any other one. One way would be remodel/train again when there are so many unknown/new organization and the Precision/Recall/F Measure score too low.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is there some specific regarding large number of cases? $\endgroup$
    – mikalai
    Commented Jul 26, 2015 at 13:56
  • $\begingroup$ Not particularly useful because no specific insight was provided $\endgroup$
    – SmallChess
    Commented Oct 25, 2015 at 7:39
0
$\begingroup$

My team and I recently experienced a similar problem. We used a Random Forest to be able to predict between 10 authors with about an 82% accuracy, as the number of users went up the accuracy went down. We were then asked to try and identify a new author which we implemented by creating a dynamic threshold for votes from the individual trees in forest. With the new user identification and 10 authors we were at about 65% accurate. We were working with twitter data also, so our document sets were fairly small. I have seen documentation where other teams have simply used the count of stop words in documents and a random forest that was 80% accurate, they weren't trying to identify a new user however.

In your situation, breaking up your problem into two different problems might yield better results. First identify if the new document belongs to any of your known users and if it does then try and decide which one.

Improve this answer
$\endgroup$
0
$\begingroup$

Document authorship attribution is a valuable technique in many areas such as plagiarism detection, and lawsuit prosecution.

we have few research papers on source code authorship attribution. In these papers, we have discussed simple models such as Naive Bayes classifier and few advanced techniques such as Deep Learning. Though, our papers are specifically on source code authorship attribution, you can apply algorithms we discussed in our papers for general document classification problems.

Good Luck!

Papers

[1]. http://www.ijmlc.org/papers/50-A243.pdf

[2]. http://www.sciencedirect.com/science/article/pii/S0167865512003571

[3]. http://link.springer.com/chapter/10.1007/978-3-642-42042-9_46

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.