I have worked on building a fraud detection solution using text mining, so I understand the scenario that has led to this question. I'll talk about the approach/techniques that is to be followed to build the fraud detection solution.I'll divide it into 4 sections
- Build line of business fraud dictionary
- Fraud detection & scoring
- Tools
- Additional notes
Building Fraud Dictionary:
As a first step you'll have to identify fraud concepts with help from subject matter expert for the line of business in concern. Fraud concept is a person, characteristic, entity, or event that represents a suspicious scenario and similar concepts can be grouped together to form a concept category. Each concept is further represented by words, phrases, entities etc. The presence of these words/phrases in the document implies occurrence of fraud concept in that transaction. The end result of this exercise will be a fraud dictionary that is a repository of concepts and suspicious key words.
For example considering the data set mentioned in the question, the numeric patterns
become the concept category, and fictitious invoice numbers
and fictitiously-generated transaction amounts
are 2 different concepts. There will be a keywords/phrases that is associated to this concept which should be captured onto the fraud dictionary.
You can use NLP technique to build the dictionary.
Steps:
1) Convert the sentence to lowercase
2) Remove stopwords, these are common words found in a language. Words like for, very, and, of, are, etc, are common stop words
3) Extract n-gram i.e., a contiguous sequence of n items from a given sequence of text. simply increasing n, model can be used to store more context
4) Assign a syntactic label (noun, verb etc.)
5) Knowledge extraction from text through semantic/syntactic analysis approach i.e., try to retain words that hold higher weight in a sentence like Noun/Verb
Fraud detection & scoring:
Phrases/Keywords to be semantically matched with Phrases/Keywords from Fraud Dictionary and Fraud concepts in transactions can be identified. At this point, care needs to be taken to maintain the context (negation, positive sense) in which a keyword/phrase has been used to avoid false positives.
For each identified occurrence of Fraud concept in the transaction, a weight of 1 can be assigned. This is done for calculating a suspicious score for the transaction. Higher the score high suspecious.
Note that using a combination of larger data set with high textual content, and extensive fraud dictionary would result in higher number potentially identified.
Tools:
You can consider using OpenNLP / StanfordNLP for Part of Speech tagging. Most of the programming language have supporting library for OpenNLP/StanfordNLP. You can choose the language based on your comfort.
Additional notes:
Hope this helps.