Questions tagged [nlp]

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural language understanding, that is, enabling computers to derive meaning from human or natural language input, and others involve natural language generation.

Filter by
Sorted by
Tagged with
4 votes
2 answers
3k views

Accuracy of Stanford NER

I am performing Named Entity Recognition using Stanford NER. I have successfully trained and tested my model. Now I want to know: 1) What is the general way of measuring accuracy of NER model ?? For ...
user avatar
  • 195
6 votes
1 answer
2k views

Question and Answer Chatbot for Customer Support

I want to build a chatbot that serves as a first line customer support on a retail website. I have a large log of chat sessions between customers and support professionals that I can use. I am ...
user avatar
12 votes
3 answers
6k views

Help regarding NER in NLTK

I have been working in NLTK for a while using Python. The problem I am facing is that their is no help available on training NER in NLTK with my custom data. They have used MaxEnt and trained it on ...
user avatar
  • 195
2 votes
2 answers
538 views

Generate tags for live chat transcripts

I'm wondering if there's a way to automatically generate a list of tags for live chat transcripts without domain knowledge. I've tried applying NLP chunking to the chat transcripts and keep only the ...
user avatar
  • 21
9 votes
2 answers
2k views

Using NLP to automate the categorization of user description

I have a huge file of customer complaints about the products my company owns and I would like to do a data analysis on those descriptions and tag a category to each of them. For example: I need to ...
user avatar
  • 1,025
18 votes
2 answers
23k views

Extract most informative parts of text from documents

Are there any articles or discussions about extracting part of text that holds the most of information about current document. For example, I have a large corpus of documents from the same domain. ...
user avatar
  • 631
5 votes
2 answers
114 views

Method for solving problem with variable number of predictors

I've been toying with this idea for a while. I think there is probably some method in the text mining literature, but I haven't come across anything just right... What is/are some methods for ...
user avatar
7 votes
1 answer
2k views

Attributes extraction from unstructured product descriptions

I am trying to match new product description with the existing ones. Product description looks like this: Panasonic DMC-FX07EB digital camera silver. These are steps to be performed: Tokenize ...
user avatar
  • 171
2 votes
3 answers
2k views

Sentiment analysis using python

I have some text files containing movie reviews I need to find out whether the review is good or bad. I tried the following code but it's not working: ...
user avatar
4 votes
2 answers
1k views

How to ensemble classifier incorporating all features in python?

I am doing a text classification task(5000 essays evenly distributed by 10 labels). I explored LinearSVC and got an accuracy of 80%. Now I guess whether accuracy ...
user avatar
  • 141
4 votes
1 answer
1k views

Good-Turing Smoothing Intuition

I'm working through the Coursera NLP course by Jurafsky & Manning, and the lecture on Good-Turing smoothing struck me odd. The example given was: You are fishing (a scenario from Josh Goodman),...
user avatar
3 votes
1 answer
454 views

What is the best practice to classify category of named entity in sentence

I have 1-4 gram text data from wikipedia for 14 categories, which I am using for NE classification. I feed named entity from sentence to lucene indexer which searches named entity from these 14 ...
user avatar
  • 131
3 votes
0 answers
301 views

How does the supposed "Unified Architecture for NLP" from Collobert and Weston 2008 really works?

In this paper (here) they suppose a "unified architecture for NLP" with deep neural networks with multitask learning My problem is to understand the layered architecture in figure 1, see ...
user avatar
6 votes
1 answer
212 views

Clustering strings inside strings?

I am not sure whether I formulated the question correctly. Basically, what I want to do is: Let's suppose I have a list of 1000 strings which look like this: cvzxcvzxstringcvzcxvz ...
user avatar
6 votes
1 answer
235 views

Is it possible to identify different queries/questions in sentence?

I want to identifies different queries in sentences. Like - Who is Bill Gates and where he was born? or ...
user avatar
1 vote
2 answers
3k views

What is the state of the art in the field of NLP? [closed]

I am new to Natural Language Processing, I think NLP is a challenging field, the syntax and semantic ambiguities could cause a lot of problems. For example I think for these problems machine ...
user avatar
  • 397
27 votes
3 answers
2k views

Why are NLP and Machine Learning communities interested in deep learning?

I hope you can help me, as I have some questions on this topic. I'm new in the field of deep learning, and while I did some tutorials, I can't relate or distinguish concepts from one another.
user avatar
1 vote
1 answer
45 views

how to get the Polysemes of a word in wordnet or any other api?

how to get the Polysemes of a word in wordnet or any other api. I am looking for any api. with java any idea is appreciated?
user avatar
-1 votes
2 answers
1k views

NLP lab, linux or windows and which programming languages? [closed]

I would like to do some data mining and NLP experiments to do some research I have decided to use NLTK or related tools and software Which environment or operating system do you suggest for my ...
user avatar
  • 397
2 votes
3 answers
3k views

A tool like Matlab for NLP?

Matlab is a great tool for some mathematical experiments, Neural Networks, Image Processing ... I would like to know if there is such a comprehensive and strong tool for data manipulation and NLP ...
user avatar
  • 397
5 votes
1 answer
218 views

Named entity disambiguation contests

I am interested in the field of named entity disambiguation and want to learn more about it. I have heard that there are contests organised by various associations on these kind of research topics. ...
user avatar
  • 151
4 votes
1 answer
1k views

Improve CoreNLP POS tagger and NER tagger?

The CoreNLP parts of speech tagger and name entity recognition tagger are pretty good out of the box, but I'd like to improve the accuracy further so that the overall program runs better. To explain ...
user avatar
13 votes
2 answers
4k views

What features are generally used from Parse trees in classification process in NLP?

I am exploring different types of parse tree structures. The two widely known parse tree structures are a) Constituency based parse tree and b) Dependency based parse tree structures. I am able to ...
user avatar
  • 231
10 votes
2 answers
358 views

Extract canonical string from a list of noisy strings

I have thousands of lists of strings, and each list has about 10 strings. Most strings in a given list are very similar, though some strings are (rarely) completely unrelated to the others and some ...
user avatar
  • 201
8 votes
1 answer
787 views

Coreference Resolution for German Texts

Does anyone know a libarary for performing coreference resolution on German texts? As far as I know, OpenNLP and Stanford NLP are not able to perform coreference resolution for German Texts. The ...
user avatar
4 votes
1 answer
642 views

OpenNLP Coreference Resolution (German)

I need to do coreference resolution for German texts and I plan to use OpenNLP to perform this task. As far as I know OpenNLP coreference resolution does not support the German language. Which ...
user avatar
4 votes
2 answers
2k views

How to implement Brown Clustering Algorithm in O(|V|k^2)

I am trying to implement the Brown Clustering Algorithm. Paper details: "Class-Based n-gram Models of Natural Language" by Brown et al The algorithm is supposed to in ...
user avatar
  • 113
3 votes
1 answer
885 views

Relative merits of different open source natural language generators

Does anyone know what (from your experience) is the best open source natural language generators (NLG) out there? What are the relative merits of each? I'm looking to do sophisticated text ...
user avatar
13 votes
3 answers
3k views

Unsupervised feature learning for NER

I have implemented NER system with the use of CRF algorithm with my handcrafted features that gave quite good results. The thing is that I used lots of different features including POS tags and lemmas....
user avatar
  • 631
12 votes
3 answers
3k views

Efficient database model for storing data indexed by n-grams

I'm working on an application which requires creating a very large database of n-grams that exist in a large text corpus. I need three efficient operation types: Lookup and insertion indexed by the n-...
user avatar
  • 288
8 votes
2 answers
192 views

What are some standard ways of computing the distance between individual search queries?

I made a similar question asking about distance between "documents" (Wikipedia articles, news stories, etc.). I made this a separate question because search queries are considerably smaller than ...
user avatar
  • 791
37 votes
5 answers
11k views

What are some standard ways of computing the distance between documents?

When I say "document", I have in mind web pages like Wikipedia articles and news stories. I prefer answers giving either vanilla lexical distance metrics or state-of-the-art semantic distance metrics,...
user avatar
  • 791
30 votes
4 answers
29k views

What algorithms should I use to perform job classification based on resume data?

Note that I am doing everything in R. The problem goes as follow: Basically, I have a list of resumes (CVs). Some candidates will have work experience before and some don't. The goal here is to: ...
user avatar
20 votes
3 answers
10k views

Dataset for Named Entity Recognition on Informal Text

I'm currently searching for labeled datasets to train a model to extract named entities from informal text (something similar to tweets). Because capitalization and grammar are often lacking in the ...
user avatar
  • 2,009
7 votes
1 answer
2k views

What are the main types of NLP annotators?

I'm new to the world of text mining and have been reading up on annotators at places like the UIMA website. I'm encountering many new terms like named entity recognition, tokenizer, lemmatizer, ...
user avatar
6 votes
1 answer
2k views

Named Entity Recognition: NLTK using Regular Expression

Many times Named Entity Recognition (NER) doesn't tag consecutive NNPs as one NE. I think editing the NER to use RegexpTagger also can improve the NER. For example, consider the following input: "...
user avatar
  • 213
7 votes
1 answer
224 views

Dealing with diverse text data

I'm currently working with a dataset with a wide range of document lengths -- anywhere from a single word to a full page of text. In addition, the grammatical structure and use of punctuation varies ...
user avatar
  • 2,009
26 votes
4 answers
13k views

Word2Vec for Named Entity Recognition

I'm looking to use google's word2vec implementation to build a named entity recognition system. I've heard that recursive neural nets with back propagation through structure are well suited for named ...
user avatar
  • 2,009
21 votes
3 answers
4k views

How to grow a list of related words based on initial keywords?

I recently saw a cool feature that was once available in Google Sheets: you start by writing a few related keywords in consecutive cells, say: "blue", "green", "yellow", and it automatically generates ...
user avatar
12 votes
4 answers
9k views

How to process natural language queries?

I'm curious about natural language querying. Stanford has what looks to be a strong set of software for processing natural language. I've also seen the Apache OpenNLP library, and the General ...
user avatar
22 votes
5 answers
4k views

How to annotate text documents with meta-data?

Having a lot of text documents (in natural language, unstructured), what are the possible ways of annotating them with some semantic meta-data? For example, consider a short document: ...
user avatar
10 votes
1 answer
2k views

What is generative and discriminative model? How are they used in Natural Language Processing?

This question asks about generative vs. discriminative algorithm, but can someone give an example of the difference between these forms when applied to Natural Language Processing? How are generative ...
user avatar
  • 2,372
61 votes
6 answers
29k views

Latent Dirichlet Allocation vs Hierarchical Dirichlet Process

Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process (HDP) are both topic modeling processes. The major difference is LDA requires the specification of the number of topics, and HDP ...
user avatar
  • 2,372

1
43 44 45 46
47