Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
0
votes
1answer
575 views

How to extract entities from text using existing ontologies?

I am working on a entity extraction task and I am using Stanford CoreNLP NER. Here, I want to detect entities of type "Animal", "Building", "Imagery", etc., which are not covered in Stanford CoreNLP ...
1
vote
1answer
18 views

Given two large corpora of text from different sources, is there an accepted way to get differences in vocabulary (n-grams) between them?

Given two large corpora of text from different sources, is there an accepted way to get differences in vocabulary (n-grams) between them? That is, to get results which say that, for example, the ...
1
vote
1answer
45 views

Data Entry Automation with ML

I am working on a data entry task with approximately 6000 entries to go over. The source comes in the form of a string and can look something like this: Air Canada B737 FFS From this I can ...
5
votes
1answer
150 views

How to compute document similarities in case of source codes?

I try to detect the probability of common authorship (person, company) of different kind of source code texts (webpages, program codes). My first idea is to apply the usual NLP tools like any token ...
1
vote
1answer
30 views

Grouping domain specific words/phrases with same meaning

I am looking at NLP methods to group together words/phrases which could have the same meaning. For example, in the sentence 'the table is broken' broken could be replaced by the following words/...
0
votes
1answer
23 views

Which are the appropriate prameters for lda modeling?

I try to implement in R test for appropriate metrics for lda. Here the way I try to use LDA ...
0
votes
0answers
9 views

What options do I have for measuring similarities by using vectors generated from texts?

I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
1
vote
2answers
377 views

extraction information from resume

I have a project in machine learning in which I need to analyze a curriculum vitae. for that I have to write a python program. It uses basic techniques of Natural Language Processing like word ...
0
votes
1answer
52 views

Apply Labeled LDA on large data

I'm using a dataset contains about 1.5M document. Each document comes with some keywords describing the topics of this document(Thus multi-labelled). Each document belongs to some authors(not just one ...
2
votes
0answers
24 views

Text extraction / mining from specific templates (ML)

believe me if I say that I have read basically all the threads in this website regarding this subject. A lot of them have a similar title, but the problem is somehow different. Small context: just ...
2
votes
1answer
17 views

How to segment old digitized newspapers into articles

I'm working on a large corpus of french daily newspapers from the 19th century that have been digitized and where the data are in the form of raw OCR text files (one text file per day). In terms of ...
0
votes
1answer
31 views

How PV-DBOW works

The authors of the Paragraph Vector paper describe PV-DBOW with: 2.3. Paragraph Vector without word ordering: Distributed bag of words The above method considers the concatenation of the ...
1
vote
1answer
356 views

Tokenize text with both American and English words

I need to tokenize a corpus of abstracts from an international conference. The abstracts are usually American English but sometimes British English. Consequently, I get 2 tokens for “organization” ...
0
votes
0answers
21 views

Sentiment Analysis of News Headlines

I'm trying to do sentiment analysis of News Headlines about a particular subject mentioned in it. Initially, I used TextBlob library for sentiment analysis to ...
1
vote
1answer
79 views

DBSCAN clustering on document [updated]?

I am new in topic modeling and text clustering domain and I am trying to learn more. I would like to use the DBSCAN to cluster the text data. There are many posts and sources on how to implement the ...
6
votes
2answers
52 views

How to compare different similarity measurements in text clustering?

I have a dataset which contains vectors generated from subtitles (each column represents a genre, each row is a movie name), my purpose is to find the most similar movie titles, I want to use ...
0
votes
0answers
13 views

Clustering text documents from multiple sources

Let's say I have a set of text documents. Half of the documents are concise social media posts containing a lot of shorthand, and the other half are long news articles. Also, half of the documents ...
1
vote
1answer
35 views

Build text complexity model based on complex examples

I try to build the user specific model which predicts whether arbitrary English text is complex for particular user or not. Having the complex and easy text samples allows to build such model but what ...
1
vote
1answer
38 views

Emotional tension score in sentences

I am beginner in natural language processing and my goal is to find a way to score sentences based on their emotional tension. More specifically, I would like to know to what degree a sentence ...
1
vote
1answer
44 views

Find specific topics with topic modelling

I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better? Topic Modelling ...
0
votes
1answer
48 views

Which libraries in Python are there in NLP to tokenize the Hindi sentence?

For English language there are libraries like NLTK, CoreNLP which are used for Text Normalization, Word Tokenization and Detokenization, Sentence Splitting etc. Like English, is there any library to ...
2
votes
0answers
11 views

pros and cons of lexical vs machine learning methods for text mining

I wanted to know what are the pros and cons are of using lexical methods and machine learning methods for classifying texts based topic. I have used a simple method of mining documents related to a ...
1
vote
0answers
21 views

Grouping paragraphs of text by type

I'm trying to parse some text, and extract data from it. Typical NLP problem. However the text contains different sections, and I know that the keywords of interest are in specific sections, but all ...
1
vote
0answers
24 views

How to classify product by specific category without machine learning?

I am working on a product classification problem which I have to identify product category. Say for the category, there are 5 levels (Big / medium / small / detail / double detail) 5 million ...
1
vote
1answer
141 views

Product classification in hierarchical categories based on multiple parameters and non-standard descriptions

I want to start a machine learning project in my company and a really big pain for spend analysts is to classify the products that buyers order for maintenance, tooling, raw material and such, as the ...
0
votes
0answers
11 views
0
votes
0answers
19 views

Bert Client fails to start and give an error

I have been trying to use the BERT model by google on my local machine. I have installed the latest version on Python3 and Pip3 but when I try to start the client it throws an error. Here is the ...
0
votes
0answers
15 views

Class Size Imbalance for LDA or any other Content based analysis

I am running some content analysis studies on my dataset which has two different classes, and each class has a respective list of the document I am analyzing. I compare the LDA topic model inference ...
1
vote
1answer
21 views

Technical term for using regular expressions to classify text?

Background I'm helping a researcher programmatically classify ~123,000 US Government court case files stored in plaintext. He wants to classify the claims as either having been "approved", "denied", ...
0
votes
1answer
28 views

What is the best approach to perform information extraction from tourist reviews using NLP, DL?

I am interested in performing some information extraction from tourist reviews about different places. I have data of 50 different places and around 300-400 reviews about each of them and I would ...
1
vote
1answer
50 views

Does it make sense to use TF-IDF to extract most important tokens from a corpus?

I have a collection of documents and I'd like to extract the most important words and phrases from the entire corpus. My understanding of TF-IDF is that it is calculated per token per document, so ...
1
vote
2answers
328 views

Mining of massive datasets

Can someone answer this question: It is from an exercise in the book: Mining of massive datasets: Chapter 3: Finding Similar Itemsets What is the largest number of k-shingles a document of n ...
0
votes
0answers
9 views

Classifying Short Texts with Spatial Features

I have a dataset of short texts (like tweets) in addition there's some geographical data attached to each tweet - coordinates, whether it was made on the road, street, outside or in the building, ...
1
vote
4answers
2k views

Parameter Tuning by Cross Validation for Random Forest

I train a binary random forest classifier on scikit-learn's 20 newsgroups dataset. I want to tune the parameters and try so by gridsearch and 3-fold cross validation on the training data. Is there ...
0
votes
1answer
27 views

Classify documents using a set of known vocabularies

I have a bunch of documents that I want to classify which ones talk about soccer (unsupervised learning, I do not want to manually label the documents). One way I am thinking about is to go online ...
1
vote
1answer
46 views

How to create corpus file?

I have text data in .xlsx or .txt files. How do I convert them to a corpus file (.tab)? I would like to use the corpus file with Orange's text add-on.
0
votes
0answers
11 views

How to handle variables with text and number?

This is a post with two related questions in one. The first question is: What is the correct procedure when I have variables with different kind of information? Imagine you have a column which has ...
1
vote
1answer
19 views

Comparing files (text sources) in Orange

What is the best method to compare text files in Orange? How do I find the difference or similar between two texts and relations between main nodes?
0
votes
0answers
11 views

How to interpret the transition and feature weights in a Conditional Random Field model?

Conditional Random Fields model have been a popular method for Named Entity Recognition as it accounts for statistical dependencies between entities and can include observed features that can aid with ...
0
votes
2answers
42 views

Targeted information extraction / focused extractive summarization

I have a large collection of project manuals, each with a large number of pages. Each manual contains some form of summary paragraphs, although these are not necessarily similar in structure or format ...
2
votes
0answers
34 views

Assigning tags to posts using predefined set of tags

I want to tag the text of a post with a predefined set of tags. A post could have multiple tags such as health, addiction, etc. I want to recommend up to $5$ tags. Total of $60$ tags is present. ...
4
votes
2answers
2k views

Plots with shaded standard deviation

What tools can I use to make a visualization similar to this one? I want to have the mean be bolded and the standard deviation be shaded.
-4
votes
1answer
515 views

How can I extract news about a particular company from various websites using RODBC package in R? And perform sentiment analysis on the data? [closed]

I would like to extract news about a company from online news by using the RODBC package in R. I would then like to use the extracted data for sentiment analysis. I want to accomplish this in such a ...
1
vote
1answer
30 views

What's the fastest way to do a text analysis over user reviews on a website for a beginner? [closed]

I want to analyse user reviews for certain products as part of a research project without having to learn analytics from scratch, as my requirement is temporary. I need to do the following: The user ...
4
votes
2answers
171 views

How would you categorize email subjects to find similar emails?

I have a list of email subjects like: <XYZ> commented on <ABC> Weekly review for <Company> Your account is ready And I want to find patterns ...
-1
votes
1answer
15 views

Identifying specific words in text

Let's say I have the following text" Is that another kitten playing in the shoes in the top right? I would like my code to extract kitten from that text. Is ...
2
votes
0answers
32 views

Text Mining with Pubmed Widget Orange

When I was running the text mining I did not have an issue for 57 different searches. I was able to retrieve all of the records regardless of how many there were. Until these 2 errors popped up. I ...
1
vote
0answers
26 views

Term Co-occurence Matrix

I am doing text mining with Orange. How can I use my bag of words to create a term co-occurrence matrix? I am hoping to use this to plot some cool bigram and semantic networks.
0
votes
0answers
68 views

Extraction of skills from Resume Using Machine Learning

I have gone through the previous questions regarding 'Resume Parsing' and 'Extraction of skills'. They didn't help me as my data is not structured. The resumes I am dealing are neither properly ...