Questions tagged [nltk]

NLTK is a free, open-source natural language processing toolkit for python. It is used primarily for text processing applications and includes libraries specifically made for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.

Filter by
Sorted by
Tagged with
4
votes
1answer
1k views

Group similar words under one topic and assign them a title

I am working on a natural language processing data problem and I have selected some keywords from it as features. I want to group them under one heading. But I can't find any method or algorithm to do ...
1
vote
1answer
826 views

Tokenize text with both American and English words

I need to tokenize a corpus of abstracts from an international conference. The abstracts are usually American English but sometimes British English. Consequently, I get 2 tokens for “organization” ...
7
votes
2answers
2k views

Combining Machine Learning classifier with NLTK Vader for Sentiment Analysis

As a part of my university project, I am researching/developing a sentiment analysis model wherein I am trying to combine NLTK Vader (SentimentIntensityAnalyzer) results with a Machine Learning ...
-1
votes
1answer
499 views

Building a text extractor to extract particular type of text from unstructured text data

I have a lot of data and manually extracted annotations for the text. I was looking for any advice to automate the annotation extraction up to a good level of accuracy. Any advice is welcome.
2
votes
2answers
188 views

What can be done so that 'teacher' and 'teaches' are treated similar?

Who teaches English? Now, after tokenizing, stemming.. it gives me Who, teach, English In my list of word, I have a ...
1
vote
1answer
175 views

Extracting Part of Speech (Source and Destinations) using text mining/NLP?

I need to extract the source and destination terms from the text documents using text mining / NLP / Information Retrieval ? Example input: I am travelling from New York to London. I am ...
8
votes
6answers
11k views

NLP: What are some popular packages for multi-word tokenization?

I intend to tokenize a number of job description texts. I have tried the standard tokenization using whitespace as the delimiter. However I noticed that there are some multi-word expressions that are ...
1
vote
0answers
182 views

How to dowload Wikileaks Cable Leaks documents as text corpus?

I'd like to perform NLP analysis on Wikileaks US Diplomatic Cable Leaks documents (https://wikileaks.org/plusd/), preferably as Python's NLTK3 corpus od Mongo DB documents. I couldn't find any option ...
3
votes
1answer
6k views

How to change plot size in nltk.plot()

I'm following along the NLTK book and would like to change the size of the axes in a lexical dispersion plot: ...
2
votes
1answer
1k views

extract names in a list of names

I have been provided with a text cleaning task and I am assuming this involves some amount of natural language processing. I have a collection of names which does not have any specific pattern and ...
4
votes
2answers
6k views

NLTK Sklearn Genism Text to Topic

I aint no data scientist/machine learner. What Im Lookin for ...
1
vote
1answer
358 views

Given one language ngram model, how do I compare likelihoods of two texts of different length?

Let's say I have conditional probabilities estimates for N-grams and I want to find out which of the two sequences of different length 'looks more natural' in terms of the given model. How does one ...
-3
votes
1answer
1k views

I need to measure Performance : AUC for this code of NLTK and skLearn [closed]

The code below measures precision and recall and F-measure (source). How can I measure AUC? ...
0
votes
1answer
3k views

Filter phrases based on correspondent POS tags

I have two lists: The first one includes sentences and the second one includes parts of speech (POS) tags. I want to use POS tags as a filter and find phrases according to a pattern such as 'JJ JJ NNS'...
3
votes
2answers
90 views

How to automatically find the sentiment?

I have written a program that scrapes data from the web and I have in possession about 5k sentences which I want to analyze. Part 1: I am just starting out in data science and wanted to know if ...
28
votes
5answers
34k views

How can I get a measure of the semantic similarity of words?

What is the best way to figure out the semantic similarity of words? Word2Vec is okay, but not ideal: ...
17
votes
6answers
66k views

Similarity between two words

I'm looking for a Python library that helps me identify the similarity between two words or sentences. I will be doing Audio to Text conversion which will result in an English dictionary or non ...
8
votes
2answers
8k views

Is there an alternative to nltk in golang?

Golang is one of my favourite languages and I want to use it for a personal NLP/ML project. Is golang's ecosystem good and rich enough for this? Is there an alternative package for nltk in golang?
2
votes
1answer
3k views

Unable to load NLTK in spark using PySpark

I have install NLTK and its working fine with the following code, I running in pyspark shell ...
0
votes
1answer
1k views

How to train NLTK Sequence labeling algorithm for using custom labels/Train set?

I am trying to recognize and classify the entity types based on the IOB/Sequence labeling. I was able to use nltk.ne_chunk() which is already trained to recognize named entities using their train set....
3
votes
4answers
2k views

Sentiment Analysis of Movie Reviews using Python

I am currently doing sentiment analysis using Python. Here I am taking all the reviews from movie dataset and using Naive Bayes algorithm to predict whether the review is positive or negative. From ...
3
votes
2answers
1k views

Word analysis in Python

I have a list of documents which look like this: ...
1
vote
2answers
2k views

Extracting words belonging to a key from the text

I have a description of a product say mobile in this form - "5.5" (13.97cm) full hd 1920 x 1080 pixels ips screen gorilla glass display with 401ppi resolution - 450 nits brightness & contrast ...
3
votes
1answer
294 views

Document Categorization Problem

I'm very new to data science in general, and have been tasked with a big challenge. My organization has a lot of documents that are all sorted on document type (not binary format, but a subjectively ...
4
votes
1answer
1k views

Creating training data

My task is to classify free text originated from customer complaints about our product. I have created a Taxonomy and have around 10 different categories. I've realized that these categories include ...
5
votes
1answer
159 views

Inferring Relational Hierarchies of Words

I am new to natural language processing and I have not heard of a problem similar to mine yet. I was wondering if anyone could refer me to a method for solving my problem, or tell me how this problem ...
1
vote
1answer
768 views

NLP : Rules for chunking Verb Phrases

I have read a lot of documentation surrounding NP chunking but what about Verb Phrases? Has there been a fixed set of rules for ...
2
votes
0answers
174 views

Data categorization

I'm using Google API to categorize and predict problems with laptops (Hardware, Software, Network, Customer satisfaction, No reason). I inserted my training data and categories are very well evaluated ...
1
vote
0answers
256 views

NLP : What are some common verbs surrounding organization names in text

I am trying to come up with some rules to detect named entities, specifically company or organization names in text. I think it makes sense to focus on verbs. There are a lot of ...
3
votes
1answer
379 views

Python: validating the existence of NLTK data with database search

I need to pull the names of companies out of resumes. Thousands of them. I was thinking of using NLTK to create a list of possible companies, and then cross-referencing the list of strings with ...
8
votes
1answer
4k views

Complex Chunking with NLTK

I am trying to figure out how to use NLTK's cascading chunker as per Chapter 7 of the NLTK book. Unfortunately, I'm running into a few issues when performing non-trivial chunking measures. Let's ...

1 2
3