Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have the following keywords retrieved from a text document.
natural language processing
sequential labeling
programmable
spell checking
techniques
forensics
important issue
categorial grammar
girls
applications
Now I want to remove unwanted words such as programmable, techniques, important issue, girls, applications from this keyword list. Is there a way to automate this? Can I consider some pos-tag pattern to do this?
Please help me!
Not sure what you are exactly looking for. There are many ways to do this. One simple way is like this,
word_list = ['natural language processing', 'sequential labeling', 'programmable', 'spell checking', 'techniques', 'forensics', 'important issue', 'categorial grammar', 'girls', 'applications']
Since you said, a list of key words,
stopwords = ['programmable','techniques', 'important issue', 'girls', 'applications']
resultwords = [word for word in word_list if word.lower() not in stopwords]
result = ' '.join(resultwords)
print (result)
This will yield,
>> natural language processing sequential labeling spell checking forensics categorial grammar
Otherwise, if you just have a string of text, use split() method to put each word into a list splitting by space.
querywords = word_list.split()
resultwords = [word for word in querywords if word.lower() not in stopwords]
result = ' '.join(resultwords)
You can always warp them into a function and automate this. As I said there are many ways to do it, without knowing what have you tried, this is a simple way to do without using nltk library.
