I have a set of phrases that lead to a binary outcome (accept/reject) and I was wondering what techniques are most helpful for extracting key phrases that are most likely to determine the outcome, given that I have a training set of data that has the English-language phrase and the observed outcome.

To illustrate the idea let me give a simple example:


  • Sounds great
  • That would be great
  • That's fine

key words: great, fine


  • I'm not sure
  • I don't think so
  • No way

key words: not, don't, no

  • $\begingroup$ Counterpoint: "Not bad at all!" "Great god, this is awful." $\endgroup$ – Sleepy Miles Mar 13 at 7:08
  • $\begingroup$ Very good point, I have run into similar such issues before. I think N-gram analysis can help this, perhaps starting at 3-grams for instance so that smaller phrases don't create conflicting scenarios. $\endgroup$ – z73758437095439 Mar 13 at 16:59

There are a variety of techniques that you could use, depending on what you would like to do.

If your goal is to gain insight into the phrases that are being used in each group, then I'd recommend looking for the most frequent N-grams of different lengths that appear in each class. Here is a related stackoverflow question that shows how you can use nltk and sklearn to extract these.

If your goal is to predict the outcome (accept/reject) given a phrase, then I'd recommend setting this up as a binary classification problem. Since those phrases are quite short, you could start with a Bag of Words approach - the scikit-learn documentation for working with text data is a good example that guides you through the steps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.