3
$\begingroup$

I'm very new to data science in general, and have been tasked with a big challenge.

My organization has a lot of documents that are all sorted on document type (not binary format, but a subjectively assigned type based on content, e.g. "Contract", "Receipt", "Statement", etc...).

Generally speaking assignment of these types is done upon receipt of the documents, and is not a challenge, though we would like to remove the human element of this categorization. Similarly, there are times when there are special attributes that we would like to identify, like "Statement showing use." Thus far, this is entirely done by human intervention.

I am a python programmer, and have been looking at tools to extract the text from these docs (all PDFs, all OCR'ed and searchable) and run analysis. Research has led me to look at standard libraries like NLTK, scikit-learn and gensim. But I'm struggling to identify what would be the best methodology for categorizing newly received documents.

My research is leading me down a few paths...one is creating a Tf-iDf vector model based on a sampling of current corpa and then creating a model for an incoming document's corpus and doing a naive bayes analysis against existing models to discern which category the incoming document belongs to based on highest probability. Question 1: is this right? If so question 2 becomes what is the right programmatic methodology for accomplishing this?

The reason I bring this up at all is because most tutorials I find seem to lean toward a binary discernment of text corpa (positive vs negative, spam vs ham). I did see scikit-learn has information on multi-label classification, but I'm not sure I'm going down the right road with it. The word "classification" seems to have different meaning in document analysis than what I would want it to mean.

If this question is too vague let me know and I can edit it to be more specific.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ sounds reasonable to start with NB using a bag-of-words approach... use this as your baseline, then start improving it via tf-idf, feature engineering while exploring alternative ML algorithms. $\endgroup$
    – Brandon Loudermilk
    Mar 24, 2016 at 20:33
  • $\begingroup$ So, just to be clear, would you train a BoW model for each classification? Then create a BoW model for an incoming doc and run NB against the trained models to test? $\endgroup$
    – tadamhicks
    Mar 24, 2016 at 21:09
  • $\begingroup$ BoW is just the features you use for training and test (a vector of binary values indicating the presence/absence of a word). You can do multiclass with NB (stats.stackexchange.com/questions/142505/…). Here is another useful intro link - sebastianraschka.com/Articles/… $\endgroup$
    – Brandon Loudermilk
    Mar 24, 2016 at 21:17

1 Answer 1

Reset to default
4
$\begingroup$

Except for the OCR part, the right bundle would be pandas and sklearn.

You can check this ipython notebook which uses TfidfVectorizer and SVC Classifier.

This classifier can make one-vs-one or one-vs-the-rest multiclass predictions, and if you use the predict_proba method instead of predict, you would have the confidence level of each category.

If you're looking for performances and you don't need prediction confidence levels, you should use LinearSVC which is way faster.

Sklearn is very well documented and you will find everything you need for text classification.

Improve this answer
$\endgroup$
5
  • $\begingroup$ You can also check this code. $\endgroup$
    – Till
    Mar 25, 2016 at 20:01
  • $\begingroup$ Till, thank you. That does answer my question(s). On another note, what is the best way to prepare training data? The ipython nb uses CSV. I'm thinking I should preprocess text to clean punctuation and store text in a comma delimited way? Do you have any suggestions? I have MOUNTAINS of PDFs to extract text from. $\endgroup$
    – tadamhicks
    Mar 25, 2016 at 20:31
  • $\begingroup$ You can remove stop words for instance by adding the parameter stop_words='english' in your Vectorizer. You should also use stemming to keep the root form of the words. And for the punctuation, the vectorizer can do it for you with attributes like token_pattern (that you can customize like that for instance r"(?u)\b\w\w+\b") or strip_accents='unicode'. $\endgroup$
    – Till
    Mar 25, 2016 at 20:45
  • $\begingroup$ Till, if you're still paying attention...the link you sent to the stemming discussion makes me think it's not yet functional within sklearn. I can certainly clean my train files with nltk, but is there a stemmer within sklearn I'm missing? $\endgroup$
    – tadamhicks
    Apr 25, 2016 at 17:52
  • $\begingroup$ @tadamhicks For the stemming part you can use the nltk package. Here is an example. You can also check this discussion $\endgroup$
    – Till
    Apr 25, 2016 at 18:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.