Fraud detection use text mining

Question

I would like to find different patterns recognition algorithm to detect different type of fraud. I have 1 million unstructured text documents about the clients' information with metadata about the client name, viewers, location in the cloud.

Here are the patterns I was thinking of and the related fraud that i would like to detect:

Numeric Patterns - fictitious invoice numbers, fictitiously-generated transaction amounts.

Time Patterns - transactions occurring too regularly, activity at unusual times or dates.

Name Patterns - similar and alerted name and addresses.

Geographic Patterns - Proximity relationships between apparently unrelated entities.

What technique can I use? any keywords?

Answer 1

I have worked on building a fraud detection solution using text mining, so I understand the scenario that has led to this question. I'll talk about the approach/techniques that is to be followed to build the fraud detection solution.I'll divide it into 4 sections

---

• Build line of business fraud dictionary
• Fraud detection & scoring
• Tools
• Additional notes

---

Building Fraud Dictionary:

As a first step you'll have to identify fraud concepts with help from subject matter expert for the line of business in concern. Fraud concept is a person, characteristic, entity, or event that represents a suspicious scenario and similar concepts can be grouped together to form a concept category. Each concept is further represented by words, phrases, entities etc. The presence of these words/phrases in the document implies occurrence of fraud concept in that transaction. The end result of this exercise will be a fraud dictionary that is a repository of concepts and suspicious key words.

For example considering the data set mentioned in the question, the numeric patterns become the concept category, and fictitious invoice numbers and fictitiously-generated transaction amounts are 2 different concepts. There will be a keywords/phrases that is associated to this concept which should be captured onto the fraud dictionary.

You can use NLP technique to build the dictionary.

Steps:

1) Convert the sentence to lowercase

2) Remove stopwords, these are common words found in a language. Words like for, very, and, of, are, etc, are common stop words

3) Extract n-gram i.e., a contiguous sequence of n items from a given sequence of text. simply increasing n, model can be used to store more context

4) Assign a syntactic label (noun, verb etc.)

5) Knowledge extraction from text through semantic/syntactic analysis approach i.e., try to retain words that hold higher weight in a sentence like Noun/Verb

Fraud detection & scoring:

Phrases/Keywords to be semantically matched with Phrases/Keywords from Fraud Dictionary and Fraud concepts in transactions can be identified. At this point, care needs to be taken to maintain the context (negation, positive sense) in which a keyword/phrase has been used to avoid false positives.

For each identified occurrence of Fraud concept in the transaction, a weight of 1 can be assigned. This is done for calculating a suspicious score for the transaction. Higher the score high suspecious.

Note that using a combination of larger data set with high textual content, and extensive fraud dictionary would result in higher number potentially identified.

Tools:

You can consider using OpenNLP / StanfordNLP for Part of Speech tagging. Most of the programming language have supporting library for OpenNLP/StanfordNLP. You can choose the language based on your comfort.

Additional notes:

• You can refer here to get an idea of extracting concepts from sentences

• Fruther read my blog on Text mining 101 to learn more about

  • TM process overview
  • Calculate term weight (TF-IDF)
  • Similarity distance measure (Cosine)
  • Overview of key text mining techniques

Hope this helps.

Answer 2

Without more detail, it's difficult to give detailed advice, but at first glance, this seems like a textbook case for machine learning. See if you can get a similar data set with known fraudulent activity and, depending on the data, pick a machine learning approach.

Answer 3

I would suggest playing around with this in either R or Python using Scikit Learn. The text will take a bit of playing with, but is pretty straightforward to normalize and turn into a feature set. I suggest applying TFIDF to normalize each text document against the entire corpus. You will then have lots of text features that you can join with your metadata features to do novelty detection. Scikit-Learn and it's excellent User Guide are a great resource for getting started in this arena as are Mahout (Java based) and the book Mahout In Action. There are also countless packages in R to accomplish this.

One thing to note is that you seem to suggest specific ways that you want to detect fraud. This type of hunch based machine learning is known as applying heuristics and tends to perform worse than pure machine learning methods. You should just focus on using novelty detection algorithms or possibly anomaly detection algorithms and let the statistics find the fraud rather than trying to apply your own intuition.