Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
0
votes
1answer
275 views

How to auto tag texts

Suppose we have predefined list of tags Tag #1, Tag #2, ..., Tag #N and we want to assign ...
2
votes
0answers
16 views

Learning to parse elements from text and assigning them to categories

I am trying to generate a table with values parsed from unstructured text. Below are a couple of examples of possibly thousands of entries. For each entry, I would like to identify the title, assign ...
2
votes
1answer
678 views

How to give names/labels to topics in LDA [duplicate]

I want to give labels to different topics created using LDA. I don't want to do it manually. I saw some papers on automatic labeling but I am still confused. How can I use the information produced ...
5
votes
1answer
11k views

Resume Parsing - extracting skills from resume using Machine Learning

I am trying to extract a skill set of an employee from his/her resume. I have resumes stored as plain text in Database. I do not have predefined skills in this case. How should I approach this problem?...
0
votes
1answer
327 views

How to interpret Hashingvectorizer representation?

I cannot really understand the logic behind Hashingvectorizer for text feature extraction. I can follow the logic of Bag of Word or TFiDF where the features are values for all/certain words/N-grams ...
0
votes
1answer
2k views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
1
vote
1answer
587 views

Cosine similarity between two folders (1 and 2) with documents, and find the most relevant set of documents (in folder 2) for each doc (in folder 2)

I have one folder named iir, it has 500 txt files. I have another json file named video (with dictionary structure). I wish to compute: for each of the 500 txt files, find the cosine similarity with ...
1
vote
1answer
207 views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
0
votes
2answers
771 views

How to select features for Text classification problem

I am working on a problem where we need to classify user query into multiple classes. Problem: Suppose we are running a website for selling products. The ...
1
vote
0answers
34 views

Classifying variable types on a list of variables

I have a list of around 700 variables which I need to perform a variable cleanup on. What complicates things is there are different numeric codes which flag an invalid value and these differ by the ...
0
votes
1answer
24 views

How to implement a basic query management and recommendation system

I'm trying to prototype a system where given a textual query (e.g. a question), I get a list of most relevant documents/questions among a pool of available documents/questions (similar to what we see ...
0
votes
1answer
68 views

Extracting specific portions of text from poorly formatted document?

I have a corpus of text files (free books) which are poorly formatted. The goal is to extract a particular chapter (say chapter 2) from the raw text with all weird formatting removed. Some documents ...
0
votes
2answers
793 views

How to detect phrases from an English sentence.

The question is not about detecting keyphrases. It is about detecting a combination of words makes a valid phrase or not. For example, "John reads New York Times in New York." Here, the phrases ...
5
votes
1answer
8k views

Text extraction from documents using NLP or Deep Learning

I am looking for references(Papers/github projects) on how to use deep learning in a text extraction task. Recently I was given a task to extract important information from documents of similar type, ...
1
vote
0answers
111 views

One Class Classification

I was reading about One Class Classification but had two doubts - 1) How does One Class Classification work because the training data is of only a particular class so in that case, during testing, ...
0
votes
1answer
86 views

NLP - extract sentence parts related to people [closed]

Thank you for your help, I appreciate your time.
1
vote
2answers
326 views

Mining of massive datasets

Can someone answer this question: It is from an exercise in the book: Mining of massive datasets: Chapter 3: Finding Similar Itemsets What is the largest number of k-shingles a document of n ...
1
vote
3answers
394 views

Machine learning - Algorithm suggestion for my problem using NLP

I am looking for a machine learning algorithm for my problem. I have a set of sentences like, ...
7
votes
1answer
124 views

which deep learning text classifier is good for health data

I have a data set like this: ...
1
vote
0answers
84 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
0answers
52 views

Data analysis of unstructured data that have non linear Key value position [closed]

I have unstructured data consist of three excel sheet Text contents of 3 tables some how related to each other But there is no linear direct relationship between them e.g First row of table 1 is not ...
2
votes
1answer
30 views

How would knowing spammers email address improve spam detection algorithms?

I am new into Artificial Intelligence field and I am working on the classic example of Spam detection using classification. I am using Naive Bayes algorithm as well as SVM. While working on them it ...
1
vote
0answers
132 views

How to do an automated SWOT analysis with Text Mining [closed]

has somebody an idea/approach how to do text mining for a SWOT analysis? (e.g. sentiment analysis) I need to assign categories (Strengths, Weaknesses, Opportunities, Risks) to words in a document and ...
3
votes
0answers
55 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
1
vote
0answers
30 views

Would it work to label 1/5th of my data and once I learn a classifier to slowly provide the unlabeled data? [closed]

I need to classify MOOC video scripts into one of 5 classes which specify the intent of the sentence, e.g. explanation, example, ...
1
vote
0answers
2k views

How can I extract skills from a resume using python? [closed]

I am looking at how to extract a set of skills from an unstructured resume format using Python. Please note that, though I have a basic idea of NLP, I am completely new to Python.
3
votes
1answer
20 views

Are there trained neural networks, that can distinguish a book's author point from what he stands against?

Let's take some specific book to narrow this example. Atlas shrugged by Ayn Rand. When i'll be saying something like "Ayn Rand's ideas", i'll mean only those, which are clearly stated in the book. ...
2
votes
0answers
28 views

Related words extraction in text processing

I'm working on some aspcet based sentiment analysis project applied on some tweets or reviews, so far I've applied all the known preprocessing methods on my training dataset which is an XML file that ...
-2
votes
1answer
740 views

Best way to extract information from text description and match it with set of words

I have 10k records of data, each record represents a unique product(10k class labels) and its description. For example, "Coffee Maker, this product takes coffee beans and brew it, to make tasty cofe". ...
2
votes
2answers
91 views

Build train data set for natural language text classification?

I have extracted ~550 video scripts (subtitles) from 11 free courses on the Coursera platform. I have pre-processed them in terms of punctuation removal, stop words removal, tokenization, stemming and ...
1
vote
0answers
15 views

Any Hadoop Tool to Annotate Text

I have large text files on HDFS... I would like to label some text in those files to improve text analysis? Do you know of any tool like that?
0
votes
1answer
249 views

What is the proper train data format in LSTM?

I want to train a model to detect wrong word using in sentence. I have 1 million sentences(word base or char base) with different length. Each position(word or char) has a label to indicate it is ...
2
votes
1answer
161 views

Why is spam detection a classification problem and not a class modelling problem

Trying to get my feet wet with machine learning on text. The most common dataset I've seen in this space is the sms dataset with classes ham and spam. And the most common and successful approach ...
0
votes
1answer
1k views

Creating labels for Text classification using keras

I have a text file with information that needs to classified based on keywords. The text file contains many number of paragraphs. And the paragraph contains keywords that we want (lets say salary ...
1
vote
1answer
57 views

Identifying most informative (sub)words/vectors that help classify a sample

I am classifying text using fastText which is a word2vec library that can also create vectors for character level n-grams and I have successfully trained a binary classifier. Now I’d like to see what ...
4
votes
2answers
2k views

Plots with shaded standard deviation

What tools can I use to make a visualization similar to this one? I want to have the mean be bolded and the standard deviation be shaded.
1
vote
0answers
142 views

Extracting date, relation and noun phrase from text

A sentence (Segmented from a document) as below: ...
7
votes
3answers
138 views

How to plot clusters in nice a way?

I have a large text dataset clusterized. Each cluster is represented by a centroid of the vectorized texts that belong to it, the number of texts, the created date, and other parameters. I can't plot ...
1
vote
0answers
973 views

Keyword Extraction from a text followed by a key value using tensorflow

I have a pdf file that contains information . I would like to extract few key terms/phrase along with a value for example (current balance : CHF (swiss francs) 1,000...
1
vote
2answers
29 views

Determining helpful Amazon feedback

I am in an ML course and one of our tasks is to predict the helpfulness of Amazon reviews. Currently, I am doing what most people seem to . That is, a hashvectorizer(2-gram) on the text, tfidf, ...
0
votes
1answer
70 views

Measuring document cluster cohesion

I have a set of clusters which each cluster contains a list of short documents. I want to compute how coherent and cohesive each cluster is and filter out the incoherent and in-cohesive ones. I am ...
1
vote
1answer
496 views

Euclidean vs. cosine similarity

I have a text dataset which I vectorize using a tfidf technique and now in order to make a cluster analysis I am measuring distances between these vector representations. I have found that a common ...
1
vote
0answers
32 views

Find words related to high or low score

I am working on text analysis problem. Person X can log in his goals and his actions to achieve his goal. Also their score is calculated based on some formula to measure progress of the goal. For ...
1
vote
0answers
31 views

Build a relevancy scoring model of articles using NLP

I'm really new to Data Science and text mining. I want to build a relevancy scoring model. Suppose I have a bag of words (guns, military, terrorists). I also have a list of articles. I want to find if ...
-3
votes
1answer
61 views

german gunning fog index function

I would like to analyse some text and most of my Reviews are german. Does anyone know if python has a good gunning fog index function for german language? I couldnt find anything best regards
1
vote
1answer
48 views

Extract details from bibliometrics data

I have set of bibliometrics data (references). I want to extract the author names, title and the name of the conference/journal from it. Since the referencing style used by different papers vary, I am ...
5
votes
2answers
106 views

Online news classification

I am performing an online news classification. The idea is to recognize group of news of the same topic. My algorithm has these steps: 1) I go through a group of feeds from news sites and I recognize ...
2
votes
2answers
817 views

How to feed my JSON dataset in Keras for character level text classification

I have a JSON dataset, for example: ...
0
votes
1answer
569 views

How to extract entities from text using existing ontologies?

I am working on a entity extraction task and I am using Stanford CoreNLP NER. Here, I want to detect entities of type "Animal", "Building", "Imagery", etc., which are not covered in Stanford CoreNLP ...
0
votes
2answers
2k views

How to alter word2vec wikipedia model for n-grams?

I have a very little data, so my word2vec model does not perform well. My intention is to identify words similar to technical terms such as 'support vector machine', 'machine learning', 'artificial ...