Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [nltk]

The Natural Language Toolkit is a Python library for computational linguistics.

1
vote
1answer
9 views
1
vote
0answers
26 views

Installing NLTK using WHL file -

I have previously used WHL (wheel) files to install various Python packages. But, it seems there's no such file for NLTK. Any workaround for this please? https://pypi.org/project/nltk/ The problem ...
1
vote
0answers
14 views

Automating scoring of answers for a given question

There is a topic 'X' on which students are asked to write an English passage. Given a question, I have multiple solutions with their scorings out of 10. For a given answer by a new candidate, one ...
0
votes
2answers
39 views

Classification: how to handle reviews/long english words in feature set with all other numerical features

I am currently working on an use case where feature set contains numeric values such as amount, as well as a review feature which contains long winded english text. the english text will very well ...
1
vote
1answer
23 views

NLP: What are some popular packages for phrase tokenization?

I'm trying to tokenize some sentences into phrases. For instance, given I think you're cute and I want to know more about you The tokens can be something like I think you're cute and I want ...
0
votes
0answers
26 views

Sentiment analysis with nltk

I'm studying sentimental analysis with python library nltk, following this example: ...
0
votes
0answers
15 views

Optimizing multinomial naive bayesian classification for large sets of text

I have a set of scientific papers that I'm wanting to classify into a number of categories. I have tagged documents to use as training and test data. I've been experimenting with multinomial naive ...
0
votes
0answers
15 views

Combining NLTK Vader with classifier

I have made a classifier for twitter data which classifies tweets into different class. To enhance the accuracy of classifier,I want to add vader setiment analysis as well. Now I have 2 ideas on ...
0
votes
0answers
12 views

Determing which GIVEN topic a question belongs to?

I have 1000 questions and each of them has to be assigned to a topic out of 'n' number of already given topics. example: Question -- Can a unique key take many NULL values? Topics -- Keys, Object,...
0
votes
0answers
27 views

Find similarity between n-gram keyword and corpus using Gensim

My problem deals with classifying a 12 million list of n-grams as either non-business related or business related keywords. The n-grams consist of 2 word n-grams, 3 word, all the way up to 8 word n-...
1
vote
1answer
22 views

sentiment analysis for multiple entry in one text

I must do sentiment analysis on a set of financial news from s&p500 for given entities (organization names), but the problem is that each news (rows in my dataset) may have more than one entity ...
0
votes
1answer
94 views

Python Code to find the number of hapax legomena in a Text or Words_List

In corpus linguistics, a hapax legomenon is a word that occurs only once within a context, either in the written record of an entire language, in the works of an author, or in a single text. The term ...
0
votes
4answers
285 views

remove special character in a List or String

Input_String is Text_Corpus of Jane Austen Book output Should be : ['to', 'be', 'or', 'not', 'to', 'be', 'that', 'is', 'the', 'question'] But getting this Output : ['to', 'be,', 'or', 'not', 'to', '...
0
votes
1answer
105 views

sentiment analysis nltk python

I'd like to perform sentiment analysis on stock comment using scikit and nltk. I already have about 100 comments on different stocks like "this stock will rock" which I marked as positive (1) or "this ...
2
votes
3answers
78 views

Why do we have to remove most common words for text analysis?

I am trying to do sentiment analysis the task is to classify racist tweets from other tweets. And I have read many articles and many have mentioned to remove the most common 10 words from the column ...
0
votes
1answer
87 views

Where to know the list of NLTK tagset?

I need to know what the abbreviations mean, e.g 'VBD', 'NN', etc. I am learning text processing and now I want to get verbs in the sentences, so that I need to know which abbreviations represent any ...
1
vote
0answers
88 views

NLP how to go beyond simple intent finding--using context and targeting objects

I'm manually applying NLP rules to a chatbot. Currently, I've a simple set of rules--actions that follow certain trigger words. Ex: "Create the match on saturday." This has been working for ...
0
votes
1answer
39 views

What would be the best way to map similar ngrams

I'm trying to map similar ngrams using Wordnet and synsets. For example: elder brother and older sibling should map to the same ...
3
votes
0answers
18 views

Tuning Lexicon Sentiment-values Using Machine-learning

I'm constructing a sentiment-analysis model using the lexicon-based approach, and wondering if I can tune the weights of each word-feature in the lexicon using machine-learning. Is this achievable ...
3
votes
0answers
57 views

multiple intents for modifying an intent of a sentence?

Say I have a sentence like 'I refuse to fly' or 'I'd like to fly'. I also have a sentence like 'I don't want to sit'. When training custom intents in one of the available NLU engines (rasa/wit/luis), ...
1
vote
0answers
69 views

TextRank using BM25F

I am trying to incorporate BM25F in textrank, I found out a scoring module http://whoosh.readthedocs.io/en/latest/api/scoring.html, but am unable to implement it. Has anyone incorporated this in ...
0
votes
1answer
132 views

Is there a synset for phrasal verbs?

Is there a database of phrasal verbs of similar semantics? Eg. one where querying ‘get in touch’ would return ‘get in contact with’, among other phrasal verbs of similar meanings? If not, given a ...
0
votes
0answers
20 views

Extract/detect IDs (like flight booking ids) from text

I'm looking to extract ids from the body of an email. The ids are similar to flight booking IDs. For example, in an email, I would like to obtain the booking reference (something like MNFF3RGC or MNF-...
0
votes
0answers
6 views

Nltk-posTag , merging into one list

I wanted to find number of nouns verbs and other stuffs in a paragraph using nltk. Since there are number of paragraphs, many lists are created under one name after I tag the paragraph. Now if I ...
1
vote
2answers
74 views

NLP: To remove verb and find the match in a sentence [closed]

Is there a NLP method like stemming, lemmatisation to figure out the below? ...
1
vote
0answers
24 views

Predict words in a given corpus from jumbled incomplete characters [closed]

A complete newbie to data science. I have to predict the word using characters given. But the characters given can be in any order, and some might be missing or wrong. The problem is I get too many ...
0
votes
1answer
1k views

How to replace short words into full words from tweets using python

I am doing sentiment analysis on tweets. Most of the tweets contains short words and i want to replace them as original/full word. Suppose that tweet is: ...
0
votes
0answers
48 views

Convert Lancaster stems back to one single word based on the stem?

I performed Stemming with Lancaster algorithm from the Python NLTK library, which is the most aggressive one, afaik. Here's an online tool that does this with small texts If I have words ...
1
vote
0answers
101 views

Extracting date, relation and noun phrase from text

A sentence (Segmented from a document) as below: ...
0
votes
0answers
556 views

How to extract a relation from a Named entity recognition model using NLTK in python

Using this sample article I have created a NLTK model which is able to perform named entity recognition - ...
0
votes
0answers
18 views

What is right approach for keyword extraction for a labelled data set

My Problem statement: I have a corpus of user comments, which are labelled with a sentiment of 1 or 0. I am trying to extra the keywords that would help me classify as 1 or 0. I am not interested in ...
0
votes
2answers
50 views

How to select features for clustering to detect the number of different unique products in a search result?

I am trying to use clustering to determine the number of products in a search of products. So far I am using kmeans clustering. I have run into a problem where I cannot determine good features to use. ...
0
votes
1answer
39 views

How to read Feature Based Grammar from a string

To read Context Free Grammars we can use nltk.CFG.fromstring(""" S-> NP VP`... """) But it can't read a feature based grammar which is like? ...
0
votes
1answer
149 views

Proper/Possible methods for extracting unstructured data from websites

I'm working in Python, using Scrapy, and NLTK to try to understand how I can extract data from college websites. My scraper can navigate through the university websites and find their tuition fees ...
1
vote
1answer
534 views

Why are Chunking and IOB tags necessary?

I've just come across chunking and I can't get my head around why is it necessary? I know that it is used for 'named entity recognition'. I have few questions: Why and how is Chunking helpful? Plus ...
0
votes
0answers
76 views

How to identify how survey takers answer a double barreled question?

I am analyzing the results of a survey where a survey taker is asked a, "double-barreled question." I am applying text analytics in order to answer the question, "which part of the question are survey ...
0
votes
1answer
30 views

Inappropriate stemming in nltk.stem

I was starting to learn stemming with nltk and a few words were quite inappropriately stemmed. For example:- very was stemmed to 'veri', important to 'import', once to 'onc', poorly to 'poorli' , etc....
4
votes
1answer
2k views

How to extract Question/s from document with NLTK?

How to extract Only Question/s from document with NLTK ? Can we categorise this Question into Y/N and details type answerable ? Note: I am one week old in NLTK ;-)
4
votes
1answer
612 views

Accuracy of word and sent tokenize versus custom tokenizers in nltk

The Natural Language Processing with Python book is a really good resource to understand basics of NLP. One of the chapters introduces training 'sentence segmentation' using Naive Bayes Classifer and ...
0
votes
1answer
973 views

Need help in improving accuracy of text classification using Naive Bayes in nltk for movie reviews

I am referring http://www.nltk.org/book/ch06.html for generating a movie review classifier. It considers all words (Nouns, adjectives, verbs..) as part of feature set. I am trying to build a ...
3
votes
2answers
176 views

What is the tag mapping for entity recognition in nltk?

When doing entity recognition using NLTK, one gets as a result a Tree with a bunch of words mapped to tags (eg. Mark -> NNP, <...
3
votes
2answers
2k views

StanfordTokenizer will be deprecated in version 3.2.5 Warning

I was testing the StanfordNERTagger using the NLTK wrapper and this warning appeared: ...
0
votes
2answers
70 views

Search the Number of occurrences of the particular words in data using Pandas.

I wanna Search how many times "bad" and "good" words are repeated in the data frame and visualize this with histogram.
2
votes
3answers
1k views

Text Classifier with multiple bag-of-words

I am training an email classifier from a dataset with separate columns for both the subject line and the content of the email itself. I've pre-processed the content column in such a way that the ...
-1
votes
0answers
423 views

Extracting place/organization name from card transaction SMS

From the credit/debit card transaction SMS (example below), I want to extract the place/organization name where the money was spent. Sample SMS Thank you for using your SBI Debit Card 607XX8310 ...
0
votes
1answer
73 views

How to go about text mining for suggestions/Tips in reviews for restaurants? [closed]

For example for restaurants reviews usually have suggestions like "Go in the evenings", "order the so and so sauce with this dish" or even "TIP: ask for the blah blah blah" How can I detect such ...
0
votes
1answer
104 views

Extracting specific information from multiple unstructured website [closed]

I want to extract specific information e.g the names of all professor teaching subject xyz from multiple college faculty directory. Each website has different structure and the content is not uniform (...
3
votes
1answer
593 views

Group similar words under one topic and assign them a title

I am working on a natural language processing data problem and I have selected some keywords from it as features. I want to group them under one heading. But I can't find any method or algorithm to do ...
1
vote
1answer
232 views

Tokenize text with both American and English words

I need to tokenize a corpus of abstracts from an international conference. The abstracts are usually American English but sometimes British English. Consequently, I get 2 tokens for “organization” ...
2
votes
0answers
511 views

Combining Machine Learning classifier with NLTK Vader for Sentiment Analysis

As a part of my university project, I am researching/developing a sentiment analysis model where in I am trying to combine NLTK Vader (SentimentIntensityAnalyzer) results with a Machine Learning ...