The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views
2
votes
1answer
21 views

Classifying dates in sentences

Let's say I have a sentences that goes like this: Hi, how are the kids. I will be going to Los Angeles next Friday and will come back the following Monday. If the date today is October 16 (...
0
votes
1answer
19 views

Information Extraction/Semantic Search for long, unstructured documents

I am stuck with a particular task of information extraction. I have a few hundred, long (5-35 pages) pdf, doc and docx project documents from which I seek to extract specific information and store ...
0
votes
0answers
9 views

How to approach the topical text categorization of a small collection of short texts?

I have a set of 200 very short documents, between 1 and 20 words each. One of my colleague would like to classify each of these documents in three predefined topics (let's call them "A", "B", and "C"...
0
votes
0answers
26 views

Collecting structured data from HTML source code: A generalized way

I am working on a task to build a generic function to extract some specific fields from HTML source code. The fields we want are such as product title, price, quantity and shipment The generic ...
0
votes
2answers
55 views

NLP to detect duplicates for very technical language

I have the following scenario, to detect duplicate products based on the description fields. The Description Field contains product technical name, dimensions, characteristics. My model needs to ...
-1
votes
0answers
8 views
0
votes
1answer
22 views

Finding cosine similarity score

I have a dataframe that looks like this: sentence intent hi greeting hello greeting buy this buy whats up conversation . . What I'd like ...
0
votes
1answer
22 views

Why is TF IDF output lognormal?

I ran a TF IDF algorithm and the result of predicted similarities using cosine similarity is a log-normal distribution. Is this a feature of the algorithm (e.g., all logit probabilities are log-...
0
votes
0answers
10 views

POS extraction using CoreNLP

I have a corpus of windows related documents, for which I need to extract nouns and verbs. However, it is required that I keep certain windows specific words such as "inline hooking", "instruction ...
1
vote
1answer
16 views

Can you suggest a better to organise the data to generate frequent itemsets?

I have a data of a bag of words in a document. The data has 3 coulumns : document number, word number, count of the word in the number. I am supposed to generate frequent itemsets of particular size. ...
0
votes
1answer
15 views

appending file text into list with file name in python

Here 20_newsgroups is a folder which contain 20 folder and each folder contain some file , i just want to make datasets containind words and file name its gives Error ...
2
votes
1answer
31 views

Predict votes of future comments

I have a database with a lot of comments, each comment has a vote, a vote can be positive or negative. ex : -2, -5, -90, +45, +20... So based on this training dataset I want to predict votes of ...
0
votes
0answers
8 views

Orange Error During Installing Text Add On [migrated]

I am trying to install the Text add-on (version 0.7.3) for Orange (version 3.23) on my Win10, but I am getting the following error during the building "ufal_udpipe": ...
1
vote
1answer
24 views

Given two large corpora of text from different sources, is there an accepted way to get differences in vocabulary (n-grams) between them?

Given two large corpora of text from different sources, is there an accepted way to get differences in vocabulary (n-grams) between them? That is, to get results which say that, for example, the ...
1
vote
1answer
17 views

Infer family type, size from reviews

I have a bunch of reviews: ...
1
vote
2answers
510 views

extraction information from resume

I have a project in machine learning in which I need to analyze a curriculum vitae. for that I have to write a python program. It uses basic techniques of Natural Language Processing like word ...
6
votes
1answer
13k views

Resume Parsing - extracting skills from resume using Machine Learning

I am trying to extract a skill set of an employee from his/her resume. I have resumes stored as plain text in Database. I do not have predefined skills in this case. How should I approach this problem?...
1
vote
2answers
300 views

Text Mining error

I'm trying to get used to text mining using orange. I set up a very easy workflow loading a corpus (tab separated text file containing 10 entries), using a corpus viewer that shows my data correctly ...
2
votes
1answer
19 views

How to segment old digitized newspapers into articles

I'm working on a large corpus of french daily newspapers from the 19th century that have been digitized and where the data are in the form of raw OCR text files (one text file per day). In terms of ...
0
votes
1answer
148 views

What is required in Affinity Propagation

I am hoping to use affinity propagation to cluster my data using sklearn. But I came across a question whether to use a distance matrix or similarity matrix in the <...
0
votes
0answers
20 views

Probabilistic model of selecting subsets of words from documents?

Is there an existing probabilistic model that deals with the selection of subsets of words from a corpus of documents? Imagine a stack of documents where a subset of the words in each document has ...
0
votes
1answer
39 views

How PV-DBOW works

The authors of the Paragraph Vector paper describe PV-DBOW with: 2.3. Paragraph Vector without word ordering: Distributed bag of words The above method considers the concatenation of the ...
0
votes
0answers
41 views

how to calculate coherence score in topic model

I am trying to calculate coherence score in topic modeling. I am following this Github link So there I need to use the preprocessed wiki and news. I got 3 questions: if the domain that I have ...
0
votes
0answers
13 views

Can we treat sentiment score of the review text as rating of the product?

I have a review text of the different products, And I need the rating of the product. So can we use sentiment score as the rating of the product.
0
votes
1answer
127 views

What is the best way to use word2vec for bilingual text similarity?

I face a problem where I need to compute similarities over bilingual (English and French) texts. The "database" looks like this: ...
0
votes
0answers
10 views

Is it possible to build an intelligent lead classifier with just a few training units

I want to build a lead classifier for my Master Thesis and wanted to ask for an assessment of feasibility. Here are the key points: (1) We have 15 customers and about 100 opportunities of which we ...
4
votes
1answer
307 views

Find matching text from a text column

This is my first time to use Data Analytics tool to figure out a solution to a problem. I have a table with following columns ...
0
votes
1answer
27 views

Recurrent Neural Networks Over Multiple Documents Over Time

So in my head, I have an idea about what this architecture should look like, or at least behave, but I am having trouble implementing it. So let me describe the problem, and if anyone has an idea on ...
2
votes
1answer
53 views

Extract details from bibliometrics data

I have set of bibliometrics data (references). I want to extract the author names, title and the name of the conference/journal from it. Since the referencing style used by different papers vary, I am ...
1
vote
0answers
23 views

Sentiment Analysis for Q&A based reviews

I'm a self-learning ML enthusiast and I recently started learning NLP and performing Sentiment Analysis on imdb, yelp, amazon datasets(using Python). I came across a dataset where the reviews were in ...
1
vote
1answer
59 views

Identifying most informative (sub)words/vectors that help classify a sample

I am classifying text using fastText which is a word2vec library that can also create vectors for character level n-grams and I have successfully trained a binary classifier. Now I’d like to see what ...
5
votes
2answers
2k views

Clustering or classifing n-gram-based text categories

I have large set of data records looking like this: "text", "category" I extract n-grams from text (2-, 3- and 4-grams) and store count of each n-gram per ...
1
vote
1answer
251 views

Extracting sections from document based on list of keywords - Python

I am new to NLP and I would like to ask how can I extract sentences from the text based on keywords that I have using Python. I created a list of keywords which will be used to extract sentences from ...
0
votes
1answer
601 views

How to extract entities from text using existing ontologies?

I am working on a entity extraction task and I am using Stanford CoreNLP NER. Here, I want to detect entities of type "Animal", "Building", "Imagery", etc., which are not covered in Stanford CoreNLP ...
1
vote
1answer
48 views

Data Entry Automation with ML

I am working on a data entry task with approximately 6000 entries to go over. The source comes in the form of a string and can look something like this: Air Canada B737 FFS From this I can ...
5
votes
1answer
154 views

How to compute document similarities in case of source codes?

I try to detect the probability of common authorship (person, company) of different kind of source code texts (webpages, program codes). My first idea is to apply the usual NLP tools like any token ...
1
vote
1answer
32 views

Grouping domain specific words/phrases with same meaning

I am looking at NLP methods to group together words/phrases which could have the same meaning. For example, in the sentence 'the table is broken' broken could be replaced by the following words/...
0
votes
1answer
24 views

Which are the appropriate prameters for lda modeling?

I try to implement in R test for appropriate metrics for lda. Here the way I try to use LDA ...
0
votes
0answers
11 views

What options do I have for measuring similarities by using vectors generated from texts?

I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
0
votes
1answer
100 views

Apply Labeled LDA on large data

I'm using a dataset contains about 1.5M document. Each document comes with some keywords describing the topics of this document(Thus multi-labelled). Each document belongs to some authors(not just one ...
2
votes
0answers
31 views

Text extraction / mining from specific templates (ML)

believe me if I say that I have read basically all the threads in this website regarding this subject. A lot of them have a similar title, but the problem is somehow different. Small context: just ...
1
vote
1answer
391 views

Tokenize text with both American and English words

I need to tokenize a corpus of abstracts from an international conference. The abstracts are usually American English but sometimes British English. Consequently, I get 2 tokens for “organization” ...
0
votes
0answers
80 views

Sentiment Analysis of News Headlines

I'm trying to do sentiment analysis of News Headlines about a particular subject mentioned in it. Initially, I used TextBlob library for sentiment analysis to ...
1
vote
1answer
119 views

DBSCAN clustering on document [updated]?

I am new in topic modeling and text clustering domain and I am trying to learn more. I would like to use the DBSCAN to cluster the text data. There are many posts and sources on how to implement the ...
6
votes
2answers
67 views

How to compare different similarity measurements in text clustering?

I have a dataset which contains vectors generated from subtitles (each column represents a genre, each row is a movie name), my purpose is to find the most similar movie titles, I want to use ...
0
votes
0answers
15 views

Clustering text documents from multiple sources

Let's say I have a set of text documents. Half of the documents are concise social media posts containing a lot of shorthand, and the other half are long news articles. Also, half of the documents ...
1
vote
1answer
35 views

Build text complexity model based on complex examples

I try to build the user specific model which predicts whether arbitrary English text is complex for particular user or not. Having the complex and easy text samples allows to build such model but what ...
1
vote
1answer
38 views

Emotional tension score in sentences

I am beginner in natural language processing and my goal is to find a way to score sentences based on their emotional tension. More specifically, I would like to know to what degree a sentence ...
1
vote
1answer
59 views

Find specific topics with topic modelling

I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better? Topic Modelling ...