Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Natural language processing (NLP) is a field of computer science, artificial intelligence, and linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human–computer interaction. Many challenges in NLP involve natural ...

0
votes
0answers
3 views

Resource and useful tips on Transfer Learning in NLP

I have a few label data for training and testing a DNN. Main purpose of my work is to train a model which can do a binary classification of text. And for this purpose, I have around 3000 label data ...
0
votes
1answer
5 views

How to auto tag texts

Suppose we have predefined list of tags Tag #1, Tag #2, ..., Tag #N and we want to assign ...
0
votes
1answer
9 views

What would be the best way to map similar ngrams

I'm trying to map similar ngrams using Wordnet and synsets. For example: elder brother and older sibling should map to the same ...
0
votes
1answer
11 views

Topic Segmentation - should it be done in Raw, TfIdf or Semantic Space?

Let's assume we have a collection of documents and wish to perform some unsupervised topic segmentation. As always, we will perform some preprocessing (including tokenization, accent-removal, ...
0
votes
0answers
9 views

NLP research: Emotive conjugation

Is there any formal NLP research into emotive conjugation (aka Russel's conjugation)? Here are some examples of emotive conjugation by Bertrand Russell that are shown on the Wikipedia page: I am ...
2
votes
1answer
30 views

NLP - How to perform semantic analysis?

I'd like to perform a textual/sentiment analysis. I was able to analyse samples with 3 labels: (positive, neutral, negative) and I used algorithms such as SVM, ...
0
votes
1answer
15 views

I need sources of interrogative, exclamatory, and imperative sentences

I am working on accumulating a large database of labeled sentences for several projects/experiments. At present I am only using Wikipedia and Project Gutenberg as sources of data. Between these two ...
0
votes
0answers
8 views
0
votes
0answers
10 views

How to extract pre-defined knowledge points from texts? [closed]

The challenge before me is to extract data points from short unstructured text. The example I have is of a large group chat for car mechanics in a city. On the chat, mechanics can ask eachother for ...
1
vote
1answer
9 views

ImportError: cannot import name 'StanfordCoreNLPParser'

I've been trying to extract subject-predicate-object triples from sentences and found this awesome API that did just that. However, when it was written, it used the ...
1
vote
0answers
21 views

Reuters / RCV1 / RCV2 datasets

I am currently tackling a multi-label classification problem. Where can I find a benchmark comparison of model results using the datasets mentioned in the title? I am interested in other benchmarks, ...
8
votes
1answer
64 views

Under what circumstance is lemmatization not an advisble step when working with text data?

Disregarding possible computational restraints, are there general applications where lemmatization would be a counterproductive step when analyzing text data? For example, would lemmatization be ...
2
votes
1answer
100 views

PMI between lemma vs surface

I was wondering whether it's possible to compute the some sort of pointwise mutual information between lemma and its surface form. First if we assume, ...
0
votes
0answers
15 views

Why use GAN in NLG?

I am interested in the GAN recently. There are many papers that recently applied GAN to NLG. I do not know much about NLP or NLG, but I wonder why I use GAN for NLG. For better quality? Or for many ...
2
votes
1answer
32 views

What's beyond topic modeling?

I tried topic modeling (LDA, NMF) to extract insights from the data. I'm curious right now, are there other methods for unsupervised learning to cluster documents by the same or similar context? (...
0
votes
0answers
12 views

how to deal with varying output layer

i am trying to do Named Entity Recognition. So, for input, i am using entire text(converted to word level embedding as input) and out put as same length as input, but only the required entity will be ...
0
votes
1answer
20 views

How to interpret Hashingvectorizer representation?

I cannot really understand the logic behind Hashingvectorizer for text feature extraction. I can follow the logic of Bag of Word or TFiDF where the features are values for all/certain words/N-grams ...
1
vote
0answers
18 views

intent detection and slot filling using tensorflow.js

I'm still getting up to speed with machine learning, but I'm aware of the papers on joint intent detection and slot filling by Bing Liu & Ian Lane, and another by Xiaodong Zhang and Houfeng Wang - ...
3
votes
0answers
10 views

multiple intents for modifying an intent of a sentence?

Say I have a sentence like 'I refuse to fly' or 'I'd like to fly'. I also have a sentence like 'I don't want to sit'. When training custom intents in one of the available NLU engines (rasa/wit/luis), ...
0
votes
1answer
46 views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
1
vote
0answers
9 views

TextRank using BM25F

I am trying to incorporate BM25F in textrank, I found out a scoring module http://whoosh.readthedocs.io/en/latest/api/scoring.html, but am unable to implement it. Has anyone incorporated this in ...
0
votes
0answers
26 views

Progress-report related dataset

Sorry if this post is in the wrong place. I'm looking for a dataset containing qualitative progress-reports, preferably in the context of project-management. For example: ...
1
vote
0answers
18 views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
2
votes
1answer
36 views

What can I use to post process an NLP tree generated from the python library `spaCy`?

Using spaCy as the NLP engine for a chatbot, I call nlp("Where are the apples?").print_tree() and receive: ...
1
vote
1answer
41 views

Most Efficient Machine Learning Algorithm for Text Analysis

Looking at Understanding Convolutional Neural Networks for NLP, Convolutional Neural Networks (CNNs) seem to be suitable not only for image recognition, but also for NLP. Are CNNs in general the best ...
0
votes
1answer
22 views

Is there a synset for phrasal verbs?

Is there a database of phrasal verbs of similar semantics? Eg. one where querying ‘get in touch’ would return ‘get in contact with’, among other phrasal verbs of similar meanings? If not, given a ...
0
votes
2answers
51 views

How to select features for Text classification problem

I am working on a problem where we need to classify user query into multiple classes. Problem: Suppose we are running a website for selling products. The ...
0
votes
2answers
37 views

What machine learning algorithms to use for unsupervised POS tagging?

I am interested in an unsupervised approach to training a POS-tagger. Labeling is very difficult and I would like to test a tagger for my specific domain, chats, where users typically write in lower ...
0
votes
2answers
32 views

Discarding rare words when comparing texts - per text, per comparison, or per codex?

I'm trying to compare texts (read: books) using KL divergence of N-gram usage frequency. first I have to calculate the frequency of N-grams, and I see (perhaps unsurprisingly) that many of the words ...
0
votes
1answer
28 views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
2
votes
1answer
22 views

Finding the most phonetically similar word from WordNet

Besides soundex and other libraries, that take two words and determine whether they are similar, is there any way to find the most similar sounding word from WordNet, for a given word? I tried to use ...
3
votes
1answer
30 views

Is it a red flag that increasing the number of parameters makes the model less able to overfit small amounts of data?

I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~...
0
votes
1answer
28 views

Extracting specific portions of text from poorly formatted document?

I have a corpus of text files (free books) which are poorly formatted. The goal is to extract a particular chapter (say chapter 2) from the raw text with all weird formatting removed. Some documents ...
0
votes
1answer
34 views

How to detect phrases from an English sentence.

The question is not about detecting keyphrases. It is about detecting a combination of words makes a valid phrase or not. For example, "John reads New York Times in New York." Here, the phrases ...
1
vote
2answers
112 views

regex to remove repeating words in a sentence

I am new to regex. I am working on a project where i need to replace repeating words with that word. for example: I need need to learn regex regex from scratch. I need to change it to: I need ...
0
votes
1answer
38 views

Text Classification with deep learning

I have questions of users and I want to classify them automatically without manually labelling them. What deep learning method would be good for text classification just from text (so unsupervised). ...
0
votes
1answer
33 views

Chat Bot Answering based on Data Corpus Self-Training

I have created a very simple chat bot based on RASA NLU. In this case, I manually create some sample input text and create a model for using it against unknown source of input. It's fine for now. As ...
-2
votes
1answer
34 views

Rasa_Nlu SpaCy installing dependencies [closed]

I'm trying to do some intent extraction/recognition. Ive installed all dependancies (i believe) but it still gives me the error: File "C:\Users\user.spyder-py3\chatbot\Outlook\rasa_nlu\components.py"...
1
vote
1answer
57 views

How to retrain Glove Vectors on top of my own data?

I am using GloVe and gensim for my project. I have a corpus of data (let's say mydata.txt) which has new words which are not in the existing Glove. So, how do I retrain glove so that the existing pre-...
0
votes
1answer
27 views

what is the reason behind the bad outputs gained by RNN, LSTM when using GloVe pretrained model in text classification?

the problem is with the results gained for accuracy and f1 afer training our model via pretrained models such as GloVe. when I apply CNN as a classifier, the result are good as follows: ...
0
votes
0answers
9 views

Should I rescale tfidf features?

I have a dataset which contains both text and numeric features. I have encoded the text ones using the TfidfVectorizer from sklearn. I would now like to apply logistic regression to the resulting ...
1
vote
1answer
30 views

Is there any named entity reconginition algorithm trained for the french language?

I am trying to implement a utility for my mobile application to perform some actions based on user questions. I need an algorithm to extract named entities from a text string (French grammar). I have ...
0
votes
0answers
60 views

is Glove better for word similarity Skip-gram/CBOW?

While looking at the slides for lecture 2 of CS224d: Deep Learning for Natural Language Processing: Link to slides It is said in slide number 31, that count based methods (ex: LSA) for creating word ...
0
votes
1answer
27 views

Matching similar strings

I have a list of conferences on different topics, e.g. ...
1
vote
0answers
38 views

Data scraping & NLP?

I'm scraping data from Bing search results for (non-commercial purposes, of course) on Python using BeautifulSoup. I've entered an Indian dessert name, called 'rasmalai' as the word that I am focusing ...
0
votes
1answer
34 views

Information retrieval / slot filling / NLP

Excuse if this has been answered before. I need to extract features and parse from a piece of text and run some analysis. For e.g. "Plot the past 5-year sales of Apple" should give me the following ...
1
vote
0answers
24 views

Predicting a new document [closed]

I have a document, (purchase agreement) of approx. 100 pages. This document is sent from buyer to seller several times, and each time there is a negotiation. Negotiation could be anything. For eg. ...
0
votes
1answer
39 views

how to input the data set in to a word2vec by keras?

I am new in using word2vec model, as a result, I do not know how I can prepare my dataset as an input for word2vec? I have searched a lot but the datasets in tutorials were in CSV format or just one ...
0
votes
1answer
110 views

Text extraction from documents using NLP or Deep Learning

I am looking for references(Papers/github projects) on how to use deep learning in a text extraction task. Recently I was given a task to extract important information from documents of similar type, ...