0
$\begingroup$

The question is not about detecting keyphrases. It is about detecting a combination of words makes a valid phrase or not. For example,

"John reads New York Times in New York."

Here, the phrases are

New York Times

New York

Detecting keyword phrases is a Text Summarization problem, however, here it is classifying whether a combination of words make a valid phrase or not.

There have been a few algorithms we have gone through including but not limited to models.phrases – Phrase (collocation) detection of Gensim, however, we are looking for better results.

|improve this question
$\endgroup$
  • $\begingroup$ Have you tried detecting n-grams and then performing summarisation? $\endgroup$ – Kaustubh Jul 6 '18 at 9:37
  • $\begingroup$ Interesting proposition. Will try and update the results here. $\endgroup$ – Saurav Mukherjee Jul 6 '18 at 20:04
  • $\begingroup$ A common practice is to detect n-grams lets say New York, New Delhi etc. and then replace them with New_York and New_Delhi etc. so that it appears as a single token, and the summarisation algorithm treats it that way. $\endgroup$ – Kaustubh Jul 7 '18 at 2:49
2
$\begingroup$

The name for what you want to do is Named Entity Recognition (NER). There are several great software packages for this, namely:

|improve this answer
$\endgroup$
  • $\begingroup$ Not really, the phrases does not always need to be Named Entities. For example, foot_soldier. $\endgroup$ – Saurav Mukherjee Sep 5 '18 at 8:26
0
$\begingroup$

I would use Stanford Parser for recognizing the phrases. It is a tool that recognizes the grammatical structure of sentences and actually performs syntactic analysis. It has available libraries in java, python, php, Ruby, C# (I guess at least one of those will work for you). I had used it in Java couple of years back and I was very happy with its accuracy.

You can find out more about it here, and it also allows you to try it online through this link (it is currently now but it's normally working).

|improve this answer
$\endgroup$
  • $\begingroup$ I have already used StanfordNLP, want a better accuracy than that. $\endgroup$ – Saurav Mukherjee Jul 6 '18 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.