Let's work it out from the ground up. Classification (also known as categorization) is an example of supervised learning. In supervised learning you have:
- model - something that approximates internal structure in your data, enabling you to reason about it and make useful predictions (e.g. predict class of an object); normally model has parameters that you want to "learn"
- training and testing datasets - sets of objects that you use for training your model (finding good values for parameters) and further evaluating
- training and classification algorithms - first describes how to learn model from training dataset, second shows how to derive class of a new object given trained model
Now let's take a simple case of spam classification. Your training dataset is a corpus of emails + corresponding labels - "spam" or "not spam". Testing dataset has the same structure, but made from some independent emails (normally one just splits his dataset and makes, say, 9/10 of it to be used for training and 1/10 - for testing). One way to model emails is to represent each of them as a set (bag) of words. If we assume that words are independent of each other, we can use Naive Bayes classifier, that is, calculate prior probabilities for each word and each class (training algorithm) and then apply Bayes theorem to find posterior probability of a new document to belong to particular class.
So, basically we have:
raw model + training set + training algorithm -> trained model
trained model + classification algorithm + new object -> object label
Now note that we represented our objects (documents) as a bag of words. But is the only way? In fact, we can extract much more from raw text. For example, instead of words as is we can use their stems or lemmas, throw out noisy stop words, add POS tags of words, extract named entities or even explore HTML structure of the document. In fact, more general representation of a document (and, in general, any object) is a feature vector. E.g. for text:
actor, analogue, bad, burn, ..., NOUN, VERB, ADJ, ..., in_bold, ... | label
0, 0, 1, 1, ..., 5, 7, 2, ..., 2, ... | not spam
0, 1, 0, 0, ..., 3, 12, 10, ..., 0, ... | spam
Here first row is a list of possible features and subsequent rows show how many times that feature occurs in a document. E.g. in first document there's no occurrences of word "actor", 1 occurrence of word "burn", 5 nouns, 2 adjectives and 2 pieces of text in bold. Last column corresponds to a resulting class label.
Using feature vector you can incorporate any properties of your texts. Though finding good set of features may take some time.
And what about model and algorithms? Are we bound to Naive Bayes. Not at all. logistic regression, SVM, decision trees - just to mention few popular classifiers. (Note, that we say "classifier" in most cases we mean model + corresponding algorithms for training and classification).
As for implementation, you can divide task into 2 parts:
- Features extraction - transforming raw texts into feature vectors.
- Object classification - building and applying model.
First point is well worked out in many NLP libraries. Second is about machine learning, so, depending on your dataset, you can use either Weka, or MLlib.
