Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
0
votes
0answers
127 views

Feeding machine learning model with different matrix

Well my question is a general question. I tried to find some relevant information before posting my question here, but no success!. I am working on ...
0
votes
1answer
771 views

Tagging documents for doc2vec

I am working on resume parsing script. I am trying to tag documents sentences with TaggedDocument function, provided by gensim. What I have managed for now is to divide every text into sentence, put ...
0
votes
1answer
129 views

What is the best way to use word2vec for bilingual text similarity?

I face a problem where I need to compute similarities over bilingual (English and French) texts. The "database" looks like this: ...
1
vote
1answer
32 views

Grouping domain specific words/phrases with same meaning

I am looking at NLP methods to group together words/phrases which could have the same meaning. For example, in the sentence 'the table is broken' broken could be replaced by the following words/...
1
vote
1answer
37 views

For text classification that has innumerable features, how do I choose the number of neurons and layers for MLPClassifier?

In my use case of text classification (identify the author from a subset of 10 authors), I find that post all processing with trigrams, there are a 100 thousand and odd features with nearly 50k ...
0
votes
1answer
22 views

Composing phrases into a grammatically correct sentence?

I'm wondering if there exist any models which could take in an ordered list of phrases without punctuation and generate a grammatically correct sentence from it. For example, for the input: ["My dog"...
2
votes
3answers
107 views

What is a suitable loss function and evaluation metric for a classification model with large number of unbalanced target classes?

I am building a multiclass classifier to predict the "Intent" of a question. There are some 100 classes in the target variable and each target class contains an unequal proportion of observations/...
1
vote
1answer
197 views

What is NLP technique to generalize manually created rules in text?

Let's say we have a free text containing key-value entities. Example: "... patient's tumour has width 6 cm and height 5 cm" Then an expert comes, marks it as important, thus we do have the rule for ...
1
vote
0answers
28 views

Dealing with small number of examples in hierarchical text classification

I am working on a multi-class text classification problem with hierarchical classes structure: super class and sub class for every text example. What am i trying to do is: based on the text predict ...
2
votes
2answers
102 views

How can I do web scrapping of all the articles containing certain keywords mentioned in a dictionary, using Python?

I would like to pull out the title and year of the all articles published in BBC since 2010 based on some keyword search for instance [MBA, LBS, Harvard...... and so on]. How can I do this using ...
1
vote
0answers
22 views

TextRank algorithm for Web content

I am looking for an algorithm that would be able to extract meaningful keyphrases from web articles. Each article has more than 2000 words and information is structured using paragraphs, h1, h2 tags ...
1
vote
0answers
34 views

output of k-mean cluster as collection of tweets

I want to cluster some 1000 tweets using k-mean algorithm. But I don't want the correct output, I just want clustering of tweets. Suppose 1 cluster contain 300 tweets than all the contain of 300 ...
3
votes
0answers
162 views

What are the rules when extracting SVO triples from preprocessed text?

If you have some already preprocessed text that is tagged, what are the rules to extract SVO triplets if you want a triple like (word, word, word). Can you give the sentence as example and extract all ...
1
vote
1answer
129 views

Help with reusing glove word embedding pretrained model

When using pretrained GloVe.6B for embedding generation, How can I get only the top most frequently used 100000 words rather than all the 4M words in the file?
4
votes
1answer
308 views

Find matching text from a text column

This is my first time to use Data Analytics tool to figure out a solution to a problem. I have a table with following columns ...
1
vote
0answers
28 views

Common deep learning practices in NLP for text classification

Are there any articles about best modern techniques in text classification (not only for English texts, but in general)? In particular, I'm interested: what kind of text preprocessing techniques are ...
2
votes
1answer
444 views

How to use build_analyzer in sklearn feature extraction

I'm trying to get list of n-gram tokens for text Ex: 'How to use build_analyzer in sklearn feature extraction ' output :['How', 'use', 'build_analyzer', 'sklearn', 'feature', 'extraction', 'How use'...
2
votes
1answer
156 views

word/sentence alignment for English document

I have a English document, which is preprocessed into two versions. I want to align words or sentences from these two versions of document. A simple example is as below: ...
2
votes
0answers
338 views

Alternate of TF-IDF

I had used TF-IDF for text similarity but the results were not so good. I tried to implement google universal encoding (tensorflow hub). The results were satisfactory but not upto the mark. Is there ...
1
vote
0answers
58 views

Proximity distance between two strings

How can I calculate the proximity between two strings Ex: I need to find all the names in unstructured text and also need to look if there is any given string (i.e 'title') nearby Input: '...
0
votes
1answer
291 views

How to auto tag texts

Suppose we have predefined list of tags Tag #1, Tag #2, ..., Tag #N and we want to assign ...
2
votes
0answers
16 views

Learning to parse elements from text and assigning them to categories

I am trying to generate a table with values parsed from unstructured text. Below are a couple of examples of possibly thousands of entries. For each entry, I would like to identify the title, assign ...
2
votes
1answer
806 views

How to give names/labels to topics in LDA [duplicate]

I want to give labels to different topics created using LDA. I don't want to do it manually. I saw some papers on automatic labeling but I am still confused. How can I use the information produced ...
6
votes
1answer
13k views

Resume Parsing - extracting skills from resume using Machine Learning

I am trying to extract a skill set of an employee from his/her resume. I have resumes stored as plain text in Database. I do not have predefined skills in this case. How should I approach this problem?...
0
votes
1answer
386 views

How to interpret Hashingvectorizer representation?

I cannot really understand the logic behind Hashingvectorizer for text feature extraction. I can follow the logic of Bag of Word or TFiDF where the features are values for all/certain words/N-grams ...
0
votes
1answer
2k views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
1
vote
1answer
630 views

Cosine similarity between two folders (1 and 2) with documents, and find the most relevant set of documents (in folder 2) for each doc (in folder 2)

I have one folder named iir, it has 500 txt files. I have another json file named video (with dictionary structure). I wish to compute: for each of the 500 txt files, find the cosine similarity with ...
1
vote
1answer
254 views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
0
votes
2answers
941 views

How to select features for Text classification problem

I am working on a problem where we need to classify user query into multiple classes. Problem: Suppose we are running a website for selling products. The ...
1
vote
0answers
47 views

Classifying variable types on a list of variables

I have a list of around 700 variables which I need to perform a variable cleanup on. What complicates things is there are different numeric codes which flag an invalid value and these differ by the ...
0
votes
1answer
25 views

How to implement a basic query management and recommendation system

I'm trying to prototype a system where given a textual query (e.g. a question), I get a list of most relevant documents/questions among a pool of available documents/questions (similar to what we see ...
0
votes
1answer
71 views

Extracting specific portions of text from poorly formatted document?

I have a corpus of text files (free books) which are poorly formatted. The goal is to extract a particular chapter (say chapter 2) from the raw text with all weird formatting removed. Some documents ...
0
votes
2answers
943 views

How to detect phrases from an English sentence.

The question is not about detecting keyphrases. It is about detecting a combination of words makes a valid phrase or not. For example, "John reads New York Times in New York." Here, the phrases ...
5
votes
1answer
10k views

Text extraction from documents using NLP or Deep Learning

I am looking for references(Papers/github projects) on how to use deep learning in a text extraction task. Recently I was given a task to extract important information from documents of similar type, ...
1
vote
0answers
122 views

One Class Classification

I was reading about One Class Classification but had two doubts - 1) How does One Class Classification work because the training data is of only a particular class so in that case, during testing, ...
0
votes
1answer
87 views

NLP - extract sentence parts related to people [closed]

Thank you for your help, I appreciate your time.
1
vote
2answers
409 views

Mining of massive datasets

Can someone answer this question: It is from an exercise in the book: Mining of massive datasets: Chapter 3: Finding Similar Itemsets What is the largest number of k-shingles a document of n ...
1
vote
3answers
434 views

Machine learning - Algorithm suggestion for my problem using NLP

I am looking for a machine learning algorithm for my problem. I have a set of sentences like, ...
7
votes
1answer
126 views

which deep learning text classifier is good for health data

I have a data set like this: ...
1
vote
0answers
102 views

Where can I find a dataset for long sequence text chunking? [closed]

Context: I have documents with reviews of articles that have the following structure: Introduction: a description of the review, dates and metadata that will be discarded. (avg~180 words, std~30 ...
1
vote
0answers
56 views

Data analysis of unstructured data that have non linear Key value position [closed]

I have unstructured data consist of three excel sheet Text contents of 3 tables some how related to each other But there is no linear direct relationship between them e.g First row of table 1 is not ...
2
votes
1answer
35 views

How would knowing spammers email address improve spam detection algorithms?

I am new into Artificial Intelligence field and I am working on the classic example of Spam detection using classification. I am using Naive Bayes algorithm as well as SVM. While working on them it ...
1
vote
0answers
148 views

How to do an automated SWOT analysis with Text Mining [closed]

has somebody an idea/approach how to do text mining for a SWOT analysis? (e.g. sentiment analysis) I need to assign categories (Strengths, Weaknesses, Opportunities, Risks) to words in a document and ...
3
votes
0answers
55 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
1
vote
0answers
30 views

Would it work to label 1/5th of my data and once I learn a classifier to slowly provide the unlabeled data? [closed]

I need to classify MOOC video scripts into one of 5 classes which specify the intent of the sentence, e.g. explanation, example, ...
1
vote
0answers
2k views

How can I extract skills from a resume using python? [closed]

I am looking at how to extract a set of skills from an unstructured resume format using Python. Please note that, though I have a basic idea of NLP, I am completely new to Python.
3
votes
1answer
20 views

Are there trained neural networks, that can distinguish a book's author point from what he stands against?

Let's take some specific book to narrow this example. Atlas shrugged by Ayn Rand. When i'll be saying something like "Ayn Rand's ideas", i'll mean only those, which are clearly stated in the book. ...
2
votes
0answers
29 views

Related words extraction in text processing

I'm working on some aspcet based sentiment analysis project applied on some tweets or reviews, so far I've applied all the known preprocessing methods on my training dataset which is an XML file that ...
-2
votes
1answer
811 views

Best way to extract information from text description and match it with set of words

I have 10k records of data, each record represents a unique product(10k class labels) and its description. For example, "Coffee Maker, this product takes coffee beans and brew it, to make tasty cofe". ...
2
votes
2answers
105 views

Build train data set for natural language text classification?

I have extracted ~550 video scripts (subtitles) from 11 free courses on the Coursera platform. I have pre-processed them in terms of punctuation removal, stop words removal, tokenization, stemming and ...