Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
1
vote
1answer
17 views

Infer family type, size from reviews

I have a bunch of reviews: ...
0
votes
0answers
19 views

Probabilistic model of selecting subsets of words from documents?

Is there an existing probabilistic model that deals with the selection of subsets of words from a corpus of documents? Imagine a stack of documents where a subset of the words in each document has ...
0
votes
0answers
17 views

how to calculate coherence score in topic model

I am trying to calculate coherence score in topic modeling. I am following this Github link So there I need to use the preprocessed wiki and news. I got 3 questions: if the domain that I have ...
0
votes
0answers
13 views

Can we treat sentiment score of the review text as rating of the product?

I have a review text of the different products, And I need the rating of the product. So can we use sentiment score as the rating of the product.
0
votes
0answers
10 views

Is it possible to build an intelligent lead classifier with just a few training units

I want to build a lead classifier for my Master Thesis and wanted to ask for an assessment of feasibility. Here are the key points: (1) We have 15 customers and about 100 opportunities of which we ...
0
votes
1answer
40 views

NLP to detect duplicates for very technical language

I have the following scenario, to detect duplicate products based on the description fields. The Description Field contains product technical name, dimensions, characteristics. My model needs to ...
1
vote
0answers
20 views

Sentiment Analysis for Q&A based reviews

I'm a self-learning ML enthusiast and I recently started learning NLP and performing Sentiment Analysis on imdb, yelp, amazon datasets(using Python). I came across a dataset where the reviews were in ...
1
vote
1answer
21 views

Given two large corpora of text from different sources, is there an accepted way to get differences in vocabulary (n-grams) between them?

Given two large corpora of text from different sources, is there an accepted way to get differences in vocabulary (n-grams) between them? That is, to get results which say that, for example, the ...
0
votes
1answer
23 views

Which are the appropriate prameters for lda modeling?

I try to implement in R test for appropriate metrics for lda. Here the way I try to use LDA ...
0
votes
0answers
10 views

What options do I have for measuring similarities by using vectors generated from texts?

I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
2
votes
0answers
27 views

Text extraction / mining from specific templates (ML)

believe me if I say that I have read basically all the threads in this website regarding this subject. A lot of them have a similar title, but the problem is somehow different. Small context: just ...
2
votes
1answer
18 views

How to segment old digitized newspapers into articles

I'm working on a large corpus of french daily newspapers from the 19th century that have been digitized and where the data are in the form of raw OCR text files (one text file per day). In terms of ...
0
votes
0answers
48 views

Sentiment Analysis of News Headlines

I'm trying to do sentiment analysis of News Headlines about a particular subject mentioned in it. Initially, I used TextBlob library for sentiment analysis to ...
0
votes
0answers
13 views

Clustering text documents from multiple sources

Let's say I have a set of text documents. Half of the documents are concise social media posts containing a lot of shorthand, and the other half are long news articles. Also, half of the documents ...
6
votes
2answers
63 views

How to compare different similarity measurements in text clustering?

I have a dataset which contains vectors generated from subtitles (each column represents a genre, each row is a movie name), my purpose is to find the most similar movie titles, I want to use ...
2
votes
0answers
13 views

pros and cons of lexical vs machine learning methods for text mining

I wanted to know what are the pros and cons are of using lexical methods and machine learning methods for classifying texts based topic. I have used a simple method of mining documents related to a ...
1
vote
1answer
47 views

Data Entry Automation with ML

I am working on a data entry task with approximately 6000 entries to go over. The source comes in the form of a string and can look something like this: Air Canada B737 FFS From this I can ...
1
vote
0answers
23 views

Grouping paragraphs of text by type

I'm trying to parse some text, and extract data from it. Typical NLP problem. However the text contains different sections, and I know that the keywords of interest are in specific sections, but all ...
1
vote
0answers
26 views

How to classify product by specific category without machine learning?

I am working on a product classification problem which I have to identify product category. Say for the category, there are 5 levels (Big / medium / small / detail / double detail) 5 million ...
0
votes
0answers
11 views

Clustering of words based on ability to predict variation in other variables

I have a data set of about 1000 observations like so: ...
0
votes
0answers
16 views

Class Size Imbalance for LDA or any other Content based analysis

I am running some content analysis studies on my dataset which has two different classes, and each class has a respective list of the document I am analyzing. I compare the LDA topic model inference ...
1
vote
1answer
23 views

Technical term for using regular expressions to classify text?

Background I'm helping a researcher programmatically classify ~123,000 US Government court case files stored in plaintext. He wants to classify the claims as either having been "approved", "denied", ...
0
votes
0answers
9 views

Classifying Short Texts with Spatial Features

I have a dataset of short texts (like tweets) in addition there's some geographical data attached to each tweet - coordinates, whether it was made on the road, street, outside or in the building, ...
1
vote
1answer
92 views

DBSCAN clustering on document [updated]?

I am new in topic modeling and text clustering domain and I am trying to learn more. I would like to use the DBSCAN to cluster the text data. There are many posts and sources on how to implement the ...
1
vote
1answer
35 views

Build text complexity model based on complex examples

I try to build the user specific model which predicts whether arbitrary English text is complex for particular user or not. Having the complex and easy text samples allows to build such model but what ...
0
votes
0answers
13 views

How to handle variables with text and number?

This is a post with two related questions in one. The first question is: What is the correct procedure when I have variables with different kind of information? Imagine you have a column which has ...
1
vote
1answer
51 views

Find specific topics with topic modelling

I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better? Topic Modelling ...
1
vote
1answer
38 views

Emotional tension score in sentences

I am beginner in natural language processing and my goal is to find a way to score sentences based on their emotional tension. More specifically, I would like to know to what degree a sentence ...
0
votes
0answers
12 views

How to interpret the transition and feature weights in a Conditional Random Field model?

Conditional Random Fields model have been a popular method for Named Entity Recognition as it accounts for statistical dependencies between entities and can include observed features that can aid with ...
0
votes
2answers
43 views

Targeted information extraction / focused extractive summarization

I have a large collection of project manuals, each with a large number of pages. Each manual contains some form of summary paragraphs, although these are not necessarily similar in structure or format ...
0
votes
1answer
69 views

Which libraries in Python are there in NLP to tokenize the Hindi sentence?

For English language there are libraries like NLTK, CoreNLP which are used for Text Normalization, Word Tokenization and Detokenization, Sentence Splitting etc. Like English, is there any library to ...
2
votes
0answers
35 views

Assigning tags to posts using predefined set of tags

I want to tag the text of a post with a predefined set of tags. A post could have multiple tags such as health, addiction, etc. I want to recommend up to $5$ tags. Total of $60$ tags is present. ...
1
vote
1answer
30 views

What's the fastest way to do a text analysis over user reviews on a website for a beginner? [closed]

I want to analyse user reviews for certain products as part of a research project without having to learn analytics from scratch, as my requirement is temporary. I need to do the following: The user ...
0
votes
1answer
30 views

What is the best approach to perform information extraction from tourist reviews using NLP, DL?

I am interested in performing some information extraction from tourist reviews about different places. I have data of 50 different places and around 300-400 reviews about each of them and I would ...
1
vote
1answer
64 views

Does it make sense to use TF-IDF to extract most important tokens from a corpus?

I have a collection of documents and I'd like to extract the most important words and phrases from the entire corpus. My understanding of TF-IDF is that it is calculated per token per document, so ...
-1
votes
1answer
16 views

Identifying specific words in text

Let's say I have the following text" Is that another kitten playing in the shoes in the top right? I would like my code to extract kitten from that text. Is ...
2
votes
0answers
35 views

Text Mining with Pubmed Widget Orange

When I was running the text mining I did not have an issue for 57 different searches. I was able to retrieve all of the records regardless of how many there were. Until these 2 errors popped up. I ...
1
vote
0answers
47 views

Term Co-occurence Matrix

I am doing text mining with Orange. How can I use my bag of words to create a term co-occurrence matrix? I am hoping to use this to plot some cool bigram and semantic networks.
0
votes
1answer
27 views

Classify documents using a set of known vocabularies

I have a bunch of documents that I want to classify which ones talk about soccer (unsupervised learning, I do not want to manually label the documents). One way I am thinking about is to go online ...
0
votes
0answers
91 views

Extraction of skills from Resume Using Machine Learning

I have gone through the previous questions regarding 'Resume Parsing' and 'Extraction of skills'. They didn't help me as my data is not structured. The resumes I am dealing are neither properly ...
1
vote
1answer
22 views

Comparing files (text sources) in Orange

What is the best method to compare text files in Orange? How do I find the difference or similar between two texts and relations between main nodes?
1
vote
0answers
32 views

Use sentiment analysis on word level (aspect level)

I am trying to use Orange 3.20 Text Mining nodes to classify sentiments using a lexicon approach (either using the Method from Liu Hu or Vader). After selecting the columns, I can see the documents ...
0
votes
0answers
17 views

Wordnet Similarity between 2 Synsets in java

Looking for appropriate Java Word net library to find distance/Similarity between 2 synsets.Currently making use of Jwnl, for retrieving Synsets. Any suggestion on how to find the Similarity measure ...
3
votes
4answers
52 views

Best way to combine two similar document

I have f.ex.: two news-articles that report the same event. However, these two text are similar BUT not the same. I would like to combine these two texts creating one text that contains only the most "...
1
vote
1answer
49 views

How to create corpus file?

I have text data in .xlsx or .txt files. How do I convert them to a corpus file (.tab)? I would like to use the corpus file with Orange's text add-on.
1
vote
0answers
35 views

R: Using Stanford Corenlp to extract sentence from original text

I am creating a summarizer in R of news articles. Hence, I am building an algorithm to get total score of each sentence based on sum of frequency of lemma in the sentences of the article, annotated ...
0
votes
0answers
13 views

Determine learned topics in text

I have a large number of texts (each about 1000 words). Every text contains a various number of topics, usually expressed as a sentence. I've extracted and categorize 22 topics and found 5000 of them ...
2
votes
0answers
22 views

identifying the primary and secondary keywords in sentense

want to identify the primary and secondary keywords which are having an impact to sentences or comparison between 2 keywords. below is the example India and China has highest population in the ...
5
votes
1answer
60 views

Text extraction from large pool of documents of different formats

I have a collection of 6 million documents stored on a hard drive (around 500GB of data storage). Those documents contain text, tables, images and come in different formats: pdf, jpg, png, rar, vsd, ...
0
votes
0answers
89 views

Root cause analysis on text data

I am working on a project to automate the process of root cause analysis for hardware devices. The input here is a text containing the problem description (as in an e-mail) and I have access to ...