Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it ...

0
votes
0answers
7 views

Description matching between two columns

I have a DataFrame with two columns, let's call them column A and column B. Each column is identified with a unique ID and a description. Usually, each idA is in a 1-1 relation with idB. ...
0
votes
0answers
10 views

output of k-mean cluster as collection of tweets

I want to cluster some 1000 tweets using k-mean algorithm. But I don't want the correct output, I just want clustering of tweets. Suppose 1 cluster contain 300 tweets than all the contain of 300 ...
0
votes
0answers
10 views

How to assign label to un-labeld text documents

I have a bunch of text documents and I would like to assign one or more than one label to each document. I know that autoencoders can be used in a semi-supervised setting to first cluster the ...
0
votes
0answers
18 views
0
votes
0answers
27 views

hashingvectorizer vs countvectorizer

I am getting significant performance difference in classifying (junk/non-junk) a large corpus of web-scraped documents when using countvectorizer (CV) vs hashvectorizer (HV) (Overall validation ...
0
votes
0answers
5 views

Public dataset for spanish news articles with title and content [migrated]

I am wondering if there are any public datasets of news in SPANISH similar to the one here: http://research.signalmedia.co/newsir16/signal-dataset.html Which consists mostly of ENGLISH news. There ...
0
votes
0answers
25 views

Intiative detection problem

I currently try to wrap my head around a problem of detecting an initiative in a company report document. The following are the types of initiatives a company can report: health, no_poverty, ...
0
votes
0answers
16 views

What are the rules when extracting SVO triples from preprocessed text?

If you have some already preprocessed text that is tagged, what are the rules to extract SVO triplets if you want a triple like (word, word, word). Can you give the sentence as example and extract all ...
1
vote
1answer
16 views

Help with reusing glove word embedding pretrained model

When using pretrained GloVe.6B for embedding generation, How can I get only the top most frequently used 100000 words rather than all the 4M words in the file?
2
votes
1answer
36 views

Find matching text from a text column

This is my first time to use Data Analytics tool to figure out a solution to a problem. I have a table with following columns ...
0
votes
0answers
93 views

Extract variable names of formula from a document

How can I extract all of the variable names from a given research paper? For example Some applications of algebraic curves to computational vision For example, below is the text and I need to ...
0
votes
0answers
18 views

Finding recurrent sentences in semantic way

In my dataset I have 2 text columns one is for findings one is for measures. Measures are also written inside the findings column. Findings are in more detail. Each row represents one document and has ...
1
vote
0answers
14 views

Common deep learning practices in NLP for text classification

Are there any articles about best modern techniques in text classification (not only for English texts, but in general)? In particular, I'm interested: what kind of text preprocessing techniques are ...
1
vote
1answer
26 views

How to use build_analyzer in sklearn feature extraction

I'm trying to get list of n-gram tokens for text Ex: 'How to use build_analyzer in sklearn feature extraction ' output :['How', 'use', 'build_analyzer', 'sklearn', 'feature', 'extraction', 'How use'...
2
votes
1answer
19 views

word/sentence alignment for English document

I have a English document, which is preprocessed into two versions. I want to align words or sentences from these two versions of document. A simple example is as below: ...
2
votes
0answers
54 views

Alternate of TF-IDF

I had used TF-IDF for text similarity but the results were not so good. I tried to implement google universal encoding (tensorflow hub). The results were satisfactory but not upto the mark. Is there ...
1
vote
0answers
38 views

Proximity distance between two strings

How can I calculate the proximity between two strings Ex: I need to find all the names in unstructured text and also need to look if there is any given string (i.e 'title') nearby Input: '...
0
votes
0answers
8 views

Find top 3 themes or tags for a piece of user supplied text

I am using Multinomial Naïve Bayes to tag a piece of text to a theme, based on a training data set. I am following the example here: https://towardsdatascience.com/multi-class-text-classification-with-...
0
votes
1answer
22 views

How to auto tag texts

Suppose we have predefined list of tags Tag #1, Tag #2, ..., Tag #N and we want to assign ...
2
votes
0answers
12 views

Learning to parse elements from text and assigning them to categories

I am trying to generate a table with values parsed from unstructured text. Below are a couple of examples of possibly thousands of entries. For each entry, I would like to identify the title, assign ...
0
votes
0answers
19 views

Selecting data in LDAvis Rshinny app

Hi please I have this R shiny code which I would like help on how to modify it server.R library("LDAvis") library(NLP) library(tm) library("servr") library(shiny) library("lda") Load dataframe ...
1
vote
1answer
37 views

How to give names/labels to topics in LDA

I want to give labels to different topics created using LDA. I don't want to do it manually. I saw some papers on automatic labeling but I am still confused. How can I use the information produced ...
3
votes
1answer
740 views

Resume Parsing - extracting skills from resume using Machine Learning

I am trying to extract a skill set of an employee from his/her resume. I have resumes stored as plain text in Database. I do not have predefined skills in this case. How should I approach this problem?...
0
votes
0answers
13 views

How to extract event from news article and merge similar event together

TL;DR Is there a way to extract events from news and cluster similar news event together? I am following the ACE methodology currently, is that a viable way? Longer version: I am currently working ...
0
votes
1answer
48 views

How to interpret Hashingvectorizer representation?

I cannot really understand the logic behind Hashingvectorizer for text feature extraction. I can follow the logic of Bag of Word or TFiDF where the features are values for all/certain words/N-grams ...
0
votes
1answer
104 views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
0
votes
1answer
177 views

Cosine similarity between two folders (1 and 2) with documents, and find the most relevant set of documents (in folder 2) for each doc (in folder 2)

I have one folder named iir, it has 500 txt files. I have another json file named video (with dictionary structure). I wish to compute: for each of the 500 txt files, find the cosine similarity with ...
1
vote
1answer
40 views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
0
votes
2answers
129 views

How to select features for Text classification problem

I am working on a problem where we need to classify user query into multiple classes. Problem: Suppose we are running a website for selling products. The ...
1
vote
0answers
21 views

Classifying variable types on a list of variables

I have a list of around 700 variables which I need to perform a variable cleanup on. What complicates things is there are different numeric codes which flag an invalid value and these differ by the ...
0
votes
1answer
21 views

How to implement a basic query management and recommendation system

I'm trying to prototype a system where given a textual query (e.g. a question), I get a list of most relevant documents/questions among a pool of available documents/questions (similar to what we see ...
0
votes
0answers
10 views

Expanding entity extraction to certain types of named-noun phrases (Bi-grams or tri-grams)

I have a corpus of articles from business publications. I'm interesting in extraction information relating to specific types of technologies and processes, as well as just interesting phrases. ...
0
votes
1answer
34 views

Extracting specific portions of text from poorly formatted document?

I have a corpus of text files (free books) which are poorly formatted. The goal is to extract a particular chapter (say chapter 2) from the raw text with all weird formatting removed. Some documents ...
0
votes
2answers
63 views

How to detect phrases from an English sentence.

The question is not about detecting keyphrases. It is about detecting a combination of words makes a valid phrase or not. For example, "John reads New York Times in New York." Here, the phrases ...
0
votes
0answers
20 views

How to handle text encoding while text input or cleaning?

Often while data reading, cleaning or performing similar tasks; often I stumble upon error message such as UnicodeEncodeError, or while tokenising these texts, I get various errors because of encoding....
1
vote
1answer
500 views

Text extraction from documents using NLP or Deep Learning

I am looking for references(Papers/github projects) on how to use deep learning in a text extraction task. Recently I was given a task to extract important information from documents of similar type, ...
0
votes
0answers
35 views

Multiclass Text Classification of Wikipedia Articles using Deep Learning

I have a collection of Wikipedia dumps. I need to classify them in a list of categories that I have. The categories are like, Sports, Law, Music, Movie, etc. There are around 300 categories. I ...
1
vote
0answers
45 views

One Class Classification

I was reading about One Class Classification but had two doubts - 1) How does One Class Classification work because the training data is of only a particular class so in that case, during testing, ...
0
votes
1answer
49 views

NLP - extract sentence parts related to people

Would be grateful for any definitions of the problem/sources about the following problem: given one-two sentences, find out which given adjectives/verbs/partial sentences in the sentence refer to a ...
1
vote
1answer
94 views

Mining of massive datasets

Can someone answer this question: It is from an exercise in the book: Mining of massive datasets: Chapter 3: Finding Similar Itemsets What is the largest number of k-shingles a document of n ...
0
votes
0answers
29 views

Locality Sensitive Hashing for two sets of Documents

I have found implementations of Locality Sensitive Hashing based on Cosine Similarity But They all deal with finding cosine similarity on a single set of documents. But I want to compare one set of ...
0
votes
0answers
31 views

Algorithm to learn and predict typographical errors/other patterns

I am having transaction data, which passes from one channel to other channel. Now in that process, the data may be subjected to some spelling mistakes or typographical errors. For eg suppose data from ...
1
vote
2answers
151 views

Machine learning - Algorithm suggestion for my problem using NLP

I am looking for a machine learning algorithm for my problem. I have a set of sentences like, ...
7
votes
1answer
98 views
1
vote
0answers
37 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
0answers
34 views

Data analysis of unstructured data that have non linear Key value position [closed]

I have unstructured data consist of three excel sheet Text contents of 3 tables some how related to each other But there is no linear direct relationship between them e.g First row of table 1 is not ...
2
votes
1answer
23 views

How would knowing spammers email address improve spam detection algorithms?

I am new into Artificial Intelligence field and I am working on the classic example of Spam detection using classification. I am using Naive Bayes algorithm as well as SVM. While working on them it ...
1
vote
0answers
36 views

How to do an automated SWOT analysis with Text Mining [closed]

has somebody an idea/approach how to do text mining for a SWOT analysis? (e.g. sentiment analysis) I need to assign categories (Strengths, Weaknesses, Opportunities, Risks) to words in a document and ...
0
votes
0answers
5 views

How to identify CS related algorithms and research areas effectively?

Given a text I want to accurately identify and classify the algorithms (e.g., support vector machines) and research areas (e.g., computer vision). I am just wondering if we can utilise knowlege-bases ...
3
votes
0answers
33 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...