• I'm helping a researcher programmatically classify ~123,000 US Government court case files stored in plaintext.
  • He wants to classify the claims as either having been "approved", "denied", or "remanded".
  • Each file has a section starting with the string "ORDER" followed by some sentences that explain the decision.
  • I suggested that we use regular expressions to:
    1. extract the "ORDER" section,
    2. separate the section into separate orders (as each case can involve multiple claims),
    3. extract the claim from each order, and
    4. classify each claim as approved or denied depending on whether the string "approved" or "denied" appears in the order regarding that claim.

Problem / Question

  • The researcher needs to describe his method in his paper and wants to know what the correct / technical term for this method of classification would be:
    • I do want to be able to explain each step, including how you changed it to a standard format. Is there a genre of text analysis we can use to classify your ultimate regular expressions approach? It would certainly be data mining, right?

  • 1
    $\begingroup$ This kind of general process can be called text mining, text extraction or data extraction (in this case features extraction wouldn't fit because it's not only about the features). $\endgroup$ – Erwan Jul 8 '19 at 13:22
  • $\begingroup$ can this dataset will be avalable.. $\endgroup$ – Dinesh Puri Feb 17 at 3:24

If the answers are stored correctly with no errors, confusion, or ambiguity then the process is a straight up search and match. If there are typos, misspellings, or what-have-you, the process is a Fuzzy string search.

Here's an excellent Medium article delving into the details of a fuzzy string search.

An example where these cases might exceeded is when approved and denied are included in the same transcript. Say 5% of all your cases are such. You'd likely want to apply some machine learning classifier to correctly categorize your input, in which case, your Q might be about the depths of possible solutions.

| improve this answer | |
  • $\begingroup$ Thanks for that tip. I searched Google for "fuzzy string search" and found a Wikipedia article about it titled "Approximate string matching", maybe consider adding a link to that page in your answer: en.wikipedia.org/wiki/Approximate_string_matching $\endgroup$ – Nathan Wailes Jul 8 '19 at 19:59
  • $\begingroup$ Do you know of a Wikipedia article about the "search and match" method? $\endgroup$ – Nathan Wailes Jul 8 '19 at 20:00
  • $\begingroup$ I don't think what I'm doing is "fuzzy string searching" because I'm not accounting for possible misspellings. $\endgroup$ – Nathan Wailes Jul 8 '19 at 20:04
  • 1
    $\begingroup$ Got it. The keywords/phrase for your method are/is "classification through string matching." $\endgroup$ – Yaakov Bressler Jul 8 '19 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.