2
$\begingroup$

I'm wondering if there's a way to automatically generate a list of tags for live chat transcripts without domain knowledge. I've tried applying NLP chunking to the chat transcripts and keep only the noun phrases as tag candidates. However, this approach would generate too many useless noun phrases. I could use some rules to prune out some of them, but it would be hard to generalize the rules.

$\endgroup$
2
$\begingroup$

You can try RAKE(Rapid Automatic Keyword Extraction) and there is a python implementation here. RAKE is an document-oriented keyword extraction algorithm and also language-independent(theoretically, since RAKE use a generated stop word list to partition candidate keywords, and considering different languages, we need to find a better way to generated stop word list.). However, about English documents, RAKE can extract keywords(or tags) in a acceptable precision and recall. RAKE is also efficient, because to use it we don't have to training a whole corpus, RAKE can generate a keyword list by calculating the word's degree and frequency then comes up a score for every candidate keyword then pick the top N words.

Hope this answer helps you or lead you a direction for your next step investigation.

| improve this answer | |
$\endgroup$
0
$\begingroup$

If you have existing properly tagged chat transcripts, you can try treating it as a supervised learning problem. If you're starting from a blank slate, that won't work.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.