I am trying to classify a large-ish number of small strings (millions) into about 10 disjunct categories. Examples of classes and strings for each class include:
email: "firstname.lastname@example.org" phone: "55", "22334455" personName: "John", "Q.", "Public" organizationName: "Reuters", "IBM" date: "Dec.", "22.10.2010" nameAndEmail: "Larsonlarson@gmail.com" (a last name has been concatenated with an email address.) phoneAndEmail: "email@example.com" (part of a phone number has been concatenated with an emal address) separator: ",", "and", "or" other: stuff that does not fit in other categories.
Carrying out classification via a set of heuristics seems tedious, so I have been experimenting with an SVM. I'm not able to get more than about 80% accuracy without doing a lot of work manually encoding features like "has an ampersand", "is uppercase", "is mostly numeric", "has a colon in the middle", etc., which sort of defeats the purpose.
I'm wondering whether part of the challenge with using an SVM is that the "salient features" of a given string, like the '@' character, are not in a fixed position in the string, so it will not get the same feature index for each string.
Does anyone have suggestions for a more appropriate approach to this task, or can anyone recommend further reading?