0
$\begingroup$

I have two lists: The first one includes sentences and the second one includes parts of speech (POS) tags. I want to use POS tags as a filter and find phrases according to a pattern such as 'JJ JJ NNS' (adjective+adjective+noun)

How can I do that?

import re

key_list= [' techniques from nonlinear analysis and partial differential equations form the basis for these studies .', 
 ' in this case , center-manifold theory can be applied to study the ordinary differential equations which replace the nonlinear \
partial differential equations for the wave motion .', 
' differential equations are cool .']

value_list = ['NNS IN JJ NN CC JJ JJ NNS VBP DT NN IN DT NNS .',
 ' IN DT NN , JJ NN MD VB VBN TO VB DT JJ NN NNS WDT VB DT JJ JJ JJ NNS IN DT NN NN .', 
 ' JJ NNS VBP JJ .']

pattern= re.compile('JJ JJ NNS',re.M|re.DOTALL)

for value in value_list:
    value_output = pattern.findall(value)
    print(value_output)

# I want  partial differential equations and ordinary differential equations
$\endgroup$
3
$\begingroup$

Hey,

Here's how i would solve this. The problem with regex is that you don't have any index so I find it better to convert your inputs into lists of single words/tags by splitting on spaces. Then it is easier to find the pattern in the tag list and get the matching words.

words = [word for key in key_list for word in key.split()]  
tags = [tag for value in value_list for tag in value.split()]  

pattern = ['JJ', 'JJ', 'NNS']
matches = []

for i in range(len(tags)):
    if tags[i:i+len(pattern)] == pattern:
        matches.append(words[i:i+len(pattern)])

matches = [' '.join(match) for match in matches]
print(matches)

# result : ['partial differential equations', 'partial differential equations']

Also, if I'm not mistaken your pos-tagger marked "differential" as a noun (NN) in "ordinary differential equations". Nltk tagger does the job and it is really fast to implement but it is not perfect.

You can try using another tagger, I personally use the Stanford one for nlp processing.

$\endgroup$
  • $\begingroup$ It worked. Thank you. How can I add to the patterns? I will have several patterns and 'JJ', 'JJ', 'NNS' is only one of them. $\endgroup$ – Hamideh Aug 29 '16 at 15:09
  • $\begingroup$ You're welcome, to make this work for any patterns just replace the "pattern" variable : eg. pattern = ['DT', 'NN'] $\endgroup$ – NBWL Aug 29 '16 at 21:21
  • $\begingroup$ I mean I have a list of patterns such as [['JJ', 'JJ', 'NNS'], ['JJ', 'NNS']] $\endgroup$ – Hamideh Aug 30 '16 at 18:48
  • $\begingroup$ Sorry, I went away for a while.I guess you can make a function out of it with pattern, key_list and value_list as arguments and call this function in a for pattern in [['JJ', 'JJ', 'NNS'], ['JJ', 'NNS']] : $\endgroup$ – NBWL Sep 4 '16 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.