I have worked on building a fraud detection solution using text mining, so I understand the scenario that has led to this question. I'll talk about the approach/techniques that is to be followed to build the fraud detection solution.I'll divide it into 4 sections
- Build line of business fraud dictionary
- Fraud detection & scoring
- Additional notes
Building Fraud Dictionary:
As a first step you'll have to identify fraud concepts with help from subject matter expert for the line of business in concern. Fraud concept is a person, characteristic, entity, or event that represents a suspicious scenario and similar concepts can be grouped together to form a concept category. Each concept is further represented by words, phrases, entities etc. The presence of these words/phrases in the document implies occurrence of fraud concept in that transaction. The end result of this exercise will be a fraud dictionary that is a repository of concepts and suspicious key words.
For example considering the data set mentioned in the question, the
numeric patterns become the concept category, and
fictitious invoice numbers and
fictitiously-generated transaction amounts are 2 different concepts. There will be a keywords/phrases that is associated to this concept which should be captured onto the fraud dictionary.
You can use NLP technique to build the dictionary.
1) Convert the sentence to lowercase
2) Remove stopwords, these are common words found in a language. Words like for, very, and, of, are, etc, are common stop words
3) Extract n-gram i.e., a contiguous sequence of n items from a given sequence of text. simply increasing n, model can be used to store more context
4) Assign a syntactic label (noun, verb etc.)
5) Knowledge extraction from text through semantic/syntactic analysis approach i.e., try to retain words that hold higher weight in a sentence like Noun/Verb
Fraud detection & scoring:
Phrases/Keywords to be semantically matched with Phrases/Keywords from Fraud Dictionary and Fraud concepts in transactions can be identified. At this point, care needs to be taken to maintain the context (negation, positive sense) in which a keyword/phrase has been used to avoid false positives.
For each identified occurrence of Fraud concept in the transaction, a weight of 1 can be assigned. This is done for calculating a suspicious score for the transaction. Higher the score high suspecious.
Note that using a combination of larger data set with high textual content, and extensive fraud dictionary would result in higher number potentially identified.
You can consider using OpenNLP / StanfordNLP for Part of Speech tagging. Most of the programming language have supporting library for OpenNLP/StanfordNLP. You can choose the language based on your comfort.
Hope this helps.