# Detection of anomal data in the text

I work with texts where there is a dialogue between two people (a client and a call center employee, the beginning and end of each person’s phrase is not defined). My goal is to classify texts in which a call center employee names words from my list. If the texts are manually marked up, can such a classification problem be solved? Are there any tricks to solve this type of problem?

Sample data: "hello hello my name is Sam Chin I'm calling for pizza delivery Okay now check your order wait a minute Sam"

There's a lot of ways to do this, one approach is to use token-based matching. You can use this to easily find any "tokens" in the text, like names, places, or just plain words.

## Methodology

I'd recommend using Rule-based Entity Recognition in spaCy. You'll define the "rules" of what the entity looks like, here's the example from the docs where we define the following patterns to find:

• An entity type of Organization and the word Apple
• An entity type of Location and the words san and francisco

Here's that in code (live example):

from spacy.lang.en import English
from spacy.pipeline import EntityRuler

nlp = English()
ruler = EntityRuler(nlp)

# These are the rules you define, look at the docs to see what your options are.
# You don't have to use the "label", you can just look for a "pattern" if you want.
patterns = [{"label": "ORG", "pattern": "Apple"},
{"label": "GPE", "pattern": [{"LOWER": "san"}, {"LOWER": "francisco"}]}]

# The text you're searching through to find your patterns
doc = nlp("Apple is opening its first big office in San Francisco.")

# This prints out the matches
print([(ent.text, ent.label_) for ent in doc.ents])


The output of this code is: [('Apple', 'ORG'), ('San Francisco', 'GPE')]

## Usage

Thankfully spaCy has some fantastic online tools for helping you write your patterns, I highly recommend you check these links out.

1. Install spaCy
2. Evaluate if you should use rules or a model (I suggest rules but I could be wrong), and if you should use token matcher or phrase matcher