Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

The Natural Language Toolkit is a Python library for computational linguistics.

0
votes
1answer
9 views

What would be the best way to map similar ngrams

I'm trying to map similar ngrams using Wordnet and synsets. For example: elder brother and older sibling should map to the same ...
3
votes
0answers
13 views

Tuning Lexicon Sentiment-values Using Machine-learning

I'm constructing a sentiment-analysis model using the lexicon-based approach, and wondering if I can tune the weights of each word-feature in the lexicon using machine-learning. Is this achievable ...
3
votes
0answers
10 views

multiple intents for modifying an intent of a sentence?

Say I have a sentence like 'I refuse to fly' or 'I'd like to fly'. I also have a sentence like 'I don't want to sit'. When training custom intents in one of the available NLU engines (rasa/wit/luis), ...
1
vote
0answers
9 views

TextRank using BM25F

I am trying to incorporate BM25F in textrank, I found out a scoring module http://whoosh.readthedocs.io/en/latest/api/scoring.html, but am unable to implement it. Has anyone incorporated this in ...
0
votes
1answer
22 views

Is there a synset for phrasal verbs?

Is there a database of phrasal verbs of similar semantics? Eg. one where querying ‘get in touch’ would return ‘get in contact with’, among other phrasal verbs of similar meanings? If not, given a ...
0
votes
0answers
14 views

Extract/detect IDs (like flight booking ids) from text

I'm looking to extract ids from the body of an email. The ids are similar to flight booking IDs. For example, in an email, I would like to obtain the booking reference (something like MNFF3RGC or MNF-...
0
votes
0answers
6 views

Nltk-posTag , merging into one list

I wanted to find number of nouns verbs and other stuffs in a paragraph using nltk. Since there are number of paragraphs, many lists are created under one name after I tag the paragraph. Now if I ...
1
vote
2answers
33 views

NLP: To remove verb and find the match in a sentence [closed]

Is there a NLP method like stemming, lemmatisation to figure out the below? ...
1
vote
0answers
15 views

Predict words in a given corpus from jumbled incomplete characters [closed]

A complete newbie to data science. I have to predict the word using characters given. But the characters given can be in any order, and some might be missing or wrong. The problem is I get too many ...
0
votes
1answer
131 views

How to replace short words into full words from tweets using python

I am doing sentiment analysis on tweets. Most of the tweets contains short words and i want to replace them as original/full word. Suppose that tweet is: ...
0
votes
0answers
19 views

Convert Lancaster stems back to one single word based on the stem?

I performed Stemming with Lancaster algorithm from the Python NLTK library, which is the most aggressive one, afaik. Here's an online tool that does this with small texts If I have words ...
1
vote
0answers
40 views

Extracting date, relation and noun phrase from text

A sentence (Segmented from a document) as below: ...
0
votes
0answers
301 views

How to extract a relation from a Named entity recognition model using NLTK in python

Using this sample article I have created a NLTK model which is able to perform named entity recognition - ...
0
votes
0answers
13 views

What is right approach for keyword extraction for a labelled data set

My Problem statement: I have a corpus of user comments, which are labelled with a sentiment of 1 or 0. I am trying to extra the keywords that would help me classify as 1 or 0. I am not interested in ...
0
votes
2answers
44 views

How to select features for clustering to detect the number of different unique products in a search result?

I am trying to use clustering to determine the number of products in a search of products. So far I am using kmeans clustering. I have run into a problem where I cannot determine good features to use. ...
0
votes
0answers
20 views

How to read Feature Based Grammar from a string

To read Context Free Grammars we can use nltk.CFG.fromstring(""" S-> NP VP`... """) But it can't read a feature based grammar which is like? ...
0
votes
1answer
70 views

Proper/Possible methods for extracting unstructured data from websites

I'm working in Python, using Scrapy, and NLTK to try to understand how I can extract data from college websites. My scraper can navigate through the university websites and find their tuition fees ...
1
vote
1answer
224 views

Why are Chunking and IOB tags necessary?

I've just come across chunking and I can't get my head around why is it necessary? I know that it is used for 'named entity recognition'. I have few questions: Why and how is Chunking helpful? Plus ...
0
votes
0answers
7 views

How to identify how survey takers answer a double barreled question?

I am analyzing the results of a survey where a survey taker is asked a, "double-barreled question." I am applying text analytics in order to answer the question, "which part of the question are survey ...
0
votes
1answer
24 views

Inappropriate stemming in nltk.stem

I was starting to learn stemming with nltk and a few words were quite inappropriately stemmed. For example:- very was stemmed to 'veri', important to 'import', once to 'onc', poorly to 'poorli' , etc....
2
votes
1answer
979 views

How to extract Question/s from document with NLTK?

How to extract Only Question/s from document with NLTK ? Can we categorise this Question into Y/N and details type answerable ? Note: I am one week old in NLTK ;-)
1
vote
1answer
400 views

Accuracy of word and sent tokenize versus custom tokenizers in nltk

The Natural Language Processing with Python book is a really good resource to understand basics of NLP. One of the chapters introduces training 'sentence segmentation' using Naive Bayes Classifer and ...
0
votes
1answer
667 views

Need help in improving accuracy of text classification using Naive Bayes in nltk for movie reviews

I am referring http://www.nltk.org/book/ch06.html for generating a movie review classifier. It considers all words (Nouns, adjectives, verbs..) as part of feature set. I am trying to build a ...
3
votes
2answers
69 views

What is the tag mapping for entity recognition in nltk?

When doing entity recognition using NLTK, one gets as a result a Tree with a bunch of words mapped to tags (eg. Mark -> NNP, <...
3
votes
2answers
1k views

StanfordTokenizer will be deprecated in version 3.2.5 Warning

I was testing the StanfordNERTagger using the NLTK wrapper and this warning appeared: ...
0
votes
2answers
57 views

Search the Number of occurrences of the particular words in data using Pandas.

I wanna Search how many times "bad" and "good" words are repeated in the data frame and visualize this with histogram.
1
vote
3answers
650 views

Text Classifier with multiple bag-of-words

I am training an email classifier from a dataset with separate columns for both the subject line and the content of the email itself. I've pre-processed the content column in such a way that the ...
0
votes
0answers
232 views

Extracting place/organization name from card transaction SMS

From the credit/debit card transaction SMS (example below), I want to extract the place/organization name where the money was spent. Sample SMS Thank you for using your SBI Debit Card 607XX8310 ...
0
votes
1answer
57 views

How to go about text mining for suggestions/Tips in reviews for restaurants? [closed]

For example for restaurants reviews usually have suggestions like "Go in the evenings", "order the so and so sauce with this dish" or even "TIP: ask for the blah blah blah" How can I detect such ...
0
votes
1answer
80 views

Extracting specific information from multiple unstructured website [closed]

I want to extract specific information e.g the names of all professor teaching subject xyz from multiple college faculty directory. Each website has different structure and the content is not uniform (...
3
votes
1answer
335 views

Group similar words under one topic and assign them a title

I am working on a natural language processing data problem and I have selected some keywords from it as features. I want to group them under one heading. But I can't find any method or algorithm to do ...
0
votes
1answer
122 views

Tokenize text with both American and English words

I need to tokenize a corpus of abstracts from an international conference. The abstracts are usually American English but sometimes British English. Consequently, I get 2 tokens for “organization” ...
0
votes
0answers
236 views

how i can improve the clusters of topics in lda

I am working on a language that is the not english and I have scraped the data from different sources. I have done my preprocessing like punctuation removal, stop-words removal and tokenization. Now I ...
2
votes
0answers
354 views

Combining Machine Learning classifier with NLTK Vader for Sentiment Analysis

As a part of my university project, I am researching/developing a sentiment analysis model where in I am trying to combine NLTK Vader (SentimentIntensityAnalyzer) results with a Machine Learning ...
-1
votes
1answer
225 views

I want to make a text extractor for extracting particular type of text from unstructured text data.

I have a lot of data and manually extracted annotations for the text. I was looking for any advice to automate the annotation extraction up to a good level of accuracy. Any advice is welcome.
2
votes
2answers
89 views

What can be done so that 'teacher' and 'teaches' are treated similar?

Who teaches English? Now, after tokenizing, stemming.. it gives me Who, teach, English In my list of word, I have a ...
1
vote
1answer
79 views

Extracting Part of Speech (Source and Destinations) using text mining/NLP?

I need to extract the source and destination terms from the text documents using text mining / NLP / Information Retrieval ? Example input: I am travelling from New York to London. I am ...
4
votes
3answers
2k views

NLP: What are some popular packages for multi-word tokenization?

I intend to tokenize a number of job description texts. I have tried the standard tokenization using whitespace as the delimiter. However I noticed that there are some multi-word expressions that are ...
1
vote
0answers
101 views

How to dowload Wikileaks Cable Leaks documents as text corpus?

I'd like to perform NLP analysis on Wikileaks US Diplomatic Cable Leaks documents (https://wikileaks.org/plusd/), preferably as Python's NLTK3 corpus od Mongo DB documents. I couldn't find any option ...
3
votes
1answer
2k views

How to change plot size in nltk.plot()

I'm following along the NLTK book and would like to change the size of the axes in a lexical dispersion plot: ...
0
votes
0answers
62 views

Efficient way of extracting full/complete sentence from tweet

I'm struggling to extract correct syntactically sentence from a tweet, ultimate goal is to build a paragraph from tweets. All what I found is the tokenizing and parsing but my goal is not to ...
1
vote
1answer
517 views

extract names in a list of names

I have been provided with a text cleaning task and I am assuming this involves some amount of natural language processing. I have a collection of names which does not have any specific pattern and ...
3
votes
2answers
5k views

NLTK Sklearn Genism Text to Topic

I aint no data scientist/machine learner. What Im Lookin for ...
1
vote
1answer
141 views

Given one language ngram model, how do I compare likelihoods of two texts of different length?

Let's say I have conditional probabilities estimates for N-grams and I want to find out which of the two sequences of different length 'looks more natural' in terms of the given model. How does one ...
-2
votes
1answer
595 views

I need to measure Performance : AUC for this code of NLTK and skLearn [closed]

The code below measures precision and recall and F-measure (source). How can I measure AUC? ...
0
votes
1answer
1k views

Filter phrases based on correspondent POS tags

I have two lists: The first one includes sentences and the second one includes parts of speech (POS) tags. I want to use POS tags as a filter and find phrases according to a pattern such as 'JJ JJ NNS'...
1
vote
2answers
66 views

How to automatically find the sentiment?

I have written a program that scrapes data from the web and I have in possession about 5k sentences which I want to analyze. Part 1: I am just starting out in data science and wanted to know if ...
14
votes
4answers
10k views

How can I get a measure of the semantic similarity of words?

What is the best way to figure out the semantic similarity of words? Word2Vec is okay, but not ideal: ...
5
votes
4answers
16k views

Similarity between two words

I'm looking for a Python library that helps me identify the similarity between two words or sentences. I will be doing Audio to Text conversion which will result in an English dictionary or non ...
6
votes
2answers
3k views

Is there an alternative to nltk in golang?

Golang is one of my favorite languages and I wanna use it for some personal project in nlp an ml. Is golang ecosystem good and rich enough for this stuffs? Is there an alternative package for nltk in ...