Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

is there an easy way convert multiple file types into one type for text analysis [on hold]

we have multiple file types excel word PowerPoint PDF and so on is there an easy to convert them into one type for text analysis
0
votes
0answers
10 views

Clustering of words based on ability to predict variation in other variables

I have a data set of about 1000 observations like so: ...
0
votes
0answers
10 views

Bert Client fails to start and give an error

I have been trying to use the BERT model by google on my local machine. I have installed the latest version on Python3 and Pip3 but when I try to start the client it throws an error. Here is the ...
0
votes
0answers
13 views

Class Size Imbalance for LDA or any other Content based analysis

I am running some content analysis studies on my dataset which has two different classes, and each class has a respective list of the document I am analyzing. I compare the LDA topic model inference ...
1
vote
1answer
20 views

Technical term for using regular expressions to classify text?

Background I'm helping a researcher programmatically classify ~123,000 US Government court case files stored in plaintext. He wants to classify the claims as either having been "approved", "denied", ...
0
votes
0answers
9 views

Classifying Short Texts with Spatial Features

I have a dataset of short texts (like tweets) in addition there's some geographical data attached to each tweet - coordinates, whether it was made on the road, street, outside or in the building, ...
0
votes
1answer
51 views

DBSCAN clustering on document [updated]?

I am new in topic modeling and text clustering domain and I am trying to learn more. I would like to use the DBSCAN to cluster the text data. There are many posts and sources on how to implement the ...
1
vote
1answer
32 views

Build text complexity model based on complex examples

I try to build the user specific model which predicts whether arbitrary English text is complex for particular user or not. Having the complex and easy text samples allows to build such model but what ...
0
votes
0answers
10 views

How to handle variables with text and number?

This is a post with two related questions in one. The first question is: What is the correct procedure when I have variables with different kind of information? Imagine you have a column which has ...
0
votes
1answer
29 views

Find specific topics with topic modelling

I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better? Topic Modelling ...
1
vote
1answer
33 views

Emotional tension score in sentences

I am beginner in natural language processing and my goal is to find a way to score sentences based on their emotional tension. More specifically, I would like to know to what degree a sentence ...
0
votes
0answers
11 views

How to interpret the transition and feature weights in a Conditional Random Field model?

Conditional Random Fields model have been a popular method for Named Entity Recognition as it accounts for statistical dependencies between entities and can include observed features that can aid with ...
0
votes
2answers
41 views

Targeted information extraction / focused extractive summarization

I have a large collection of project manuals, each with a large number of pages. Each manual contains some form of summary paragraphs, although these are not necessarily similar in structure or format ...
0
votes
1answer
31 views

Which libraries in Python are there in NLP to tokenize the Hindi sentence?

For English language there are libraries like NLTK, CoreNLP which are used for Text Normalization, Word Tokenization and Detokenization, Sentence Splitting etc. Like English, is there any library to ...
2
votes
0answers
31 views

Assigning tags to posts using predefined set of tags

I want to tag the text of a post with a predefined set of tags. A post could have multiple tags such as health, addiction, etc. I want to recommend up to $5$ tags. Total of $60$ tags is present. ...
1
vote
1answer
30 views

What's the fastest way to do a text analysis over user reviews on a website for a beginner? [closed]

I want to analyse user reviews for certain products as part of a research project without having to learn analytics from scratch, as my requirement is temporary. I need to do the following: The user ...
0
votes
1answer
27 views

What is the best approach to perform information extraction from tourist reviews using NLP, DL?

I am interested in performing some information extraction from tourist reviews about different places. I have data of 50 different places and around 300-400 reviews about each of them and I would ...
1
vote
1answer
41 views

Does it make sense to use TF-IDF to extract most important tokens from a corpus?

I have a collection of documents and I'd like to extract the most important words and phrases from the entire corpus. My understanding of TF-IDF is that it is calculated per token per document, so ...
-1
votes
1answer
14 views

Identifying specific words in text

Let's say I have the following text" Is that another kitten playing in the shoes in the top right? I would like my code to extract kitten from that text. Is ...
2
votes
0answers
26 views

Text Mining with Pubmed Widget Orange

When I was running the text mining I did not have an issue for 57 different searches. I was able to retrieve all of the records regardless of how many there were. Until these 2 errors popped up. I ...
1
vote
0answers
20 views

Term Co-occurence Matrix

I am doing text mining with Orange. How can I use my bag of words to create a term co-occurrence matrix? I am hoping to use this to plot some cool bigram and semantic networks.
0
votes
1answer
26 views

Classify documents using a set of known vocabularies

I have a bunch of documents that I want to classify which ones talk about soccer (unsupervised learning, I do not want to manually label the documents). One way I am thinking about is to go online ...
0
votes
0answers
50 views

Extraction of skills from Resume Using Machine Learning

I have gone through the previous questions regarding 'Resume Parsing' and 'Extraction of skills'. They didn't help me as my data is not structured. The resumes I am dealing are neither properly ...
1
vote
1answer
16 views

Comparing files (text sources) in Orange

What is the best method to compare text files in Orange? How do I find the difference or similar between two texts and relations between main nodes?
1
vote
0answers
27 views

Use sentiment analysis on word level (aspect level)

I am trying to use Orange 3.20 Text Mining nodes to classify sentiments using a lexicon approach (either using the Method from Liu Hu or Vader). After selecting the columns, I can see the documents ...
0
votes
0answers
11 views

Wordnet Similarity between 2 Synsets in java

Looking for appropriate Java Word net library to find distance/Similarity between 2 synsets.Currently making use of Jwnl, for retrieving Synsets. Any suggestion on how to find the Similarity measure ...
3
votes
4answers
42 views

Best way to combine two similar document

I have f.ex.: two news-articles that report the same event. However, these two text are similar BUT not the same. I would like to combine these two texts creating one text that contains only the most "...
1
vote
1answer
35 views

How to create corpus file?

I have text data in .xlsx or .txt files. How do I convert them to a corpus file (.tab)? I would like to use the corpus file with Orange's text add-on.
1
vote
0answers
24 views

R: Using Stanford Corenlp to extract sentence from original text

I am creating a summarizer in R of news articles. Hence, I am building an algorithm to get total score of each sentence based on sum of frequency of lemma in the sentences of the article, annotated ...
0
votes
0answers
11 views

Determine learned topics in text

I have a large number of texts (each about 1000 words). Every text contains a various number of topics, usually expressed as a sentence. I've extracted and categorize 22 topics and found 5000 of them ...
2
votes
0answers
21 views

identifying the primary and secondary keywords in sentense

want to identify the primary and secondary keywords which are having an impact to sentences or comparison between 2 keywords. below is the example India and China has highest population in the ...
5
votes
1answer
52 views

Text extraction from large pool of documents of different formats

I have a collection of 6 million documents stored on a hard drive (around 500GB of data storage). Those documents contain text, tables, images and come in different formats: pdf, jpg, png, rar, vsd, ...
0
votes
0answers
43 views

Root cause analysis on text data

I am working on a project to automate the process of root cause analysis for hardware devices. The input here is a text containing the problem description (as in an e-mail) and I have access to ...
2
votes
2answers
58 views

Need help with entity tagging

I need to design a system which can identify movie and production company names in a sentence. The approach that comes to my ...
-1
votes
1answer
28 views

How can I categoriese / classify a cluster of words?

I am just wondering if it is possible to classify word clusters? For example if I provide you an array of words [bird,chicken,dock,park,apple,grapes,furits,juice] ...
1
vote
0answers
37 views

Is it a good idea to train Neural Network for classification on dataset where each document has a different class i.e. no class is repeated again?

My goal is to build a recommendation model for which I want to use Neural Network (LSTM). The user will give some input keywords and the model should return the suggestions (classes) based on ...
0
votes
0answers
10 views

Is it possible to calculate precision/recall for a bag-of-words model?

Suppose I have a list of keywords given to a document: {keyword, extract, graph, represent, text, weight, number, document} and then I have the keywords ...
2
votes
3answers
143 views

How to measure the similarity between two text documents?

Assume, I have 100 text documents, and I want to cluster those documents. The first step is the construct pairwise similarity matrix 100X100 for the documents My ...
0
votes
0answers
20 views

How PV-DBOW works

The authors of the Paragraph Vector paper describe PV-DBOW with: 2.3. Paragraph Vector without word ordering: Distributed bag of words The above method considers the concatenation of the ...
1
vote
1answer
97 views

What are CRF (Conditional Random Field)

Looking for language modeling, I have been finding CRF in a lot of places which is but looking online for the same isn't actually helping me a lot. I referred Edwin Chen's blog and Ravish Chawala's ...
0
votes
1answer
27 views

Comparing English word pronunciation complexity

I'm trying to figure out a way to compute a score for the pronunciation of a given english word, so I can use that score to compare the pronunciation complexity between english words. Eg: Given ...
0
votes
0answers
51 views

Multi-class string classification

Currently working on Resume Rarser tool using doc2vec. The main assumption that I take when parsing resume is that each line of text (docx, pdf etc) contains information of one class. Although ...
1
vote
0answers
37 views

NLP to recognize the meaning of a paragraph

How can I apply NLP to extract the summary of a paragraph,I found this but was wondering if there is a better and easy way before implementing it, as the ans seems to be an old one. Basically what I ...
1
vote
0answers
24 views

Anomaly detection in structured textual data

Pls refer screenshot for sample data. As can be seen most of the fields in data are textual and highly correlated but each row has unique values and hence won't be right to call it categorical. I ...
2
votes
0answers
122 views

Hierarchical classification with multi-class predictor for every parent node

Edit: It turned out that I had an error in my function to compute the combined probabilities (a typo that changed the behavior of my function quite a bit without giving me an error message). Without ...
1
vote
2answers
287 views

extraction information from resume

I have a project in machine learning in which I need to analyze a curriculum vitae. for that I have to write a python program. It uses basic techniques of Natural Language Processing like word ...
5
votes
3answers
139 views

How does Doc2Vec treat numerical data which is a part of text data?

I have data containing both numbers and raw text related differently like: The power of diesel generator is 15kva. I need a single phase generator. Three phase generator required of 140 kva. Need 70g/...
4
votes
2answers
344 views

Date Extraction in Python

I would like to extract all date information from a given document. Essentially, I guess this can be done with a lot of regexes: 2019-02-20 20.02.2019 ("German format") 02/2019 ("February 2019") "...
1
vote
0answers
38 views

Extracting text from few areas on product label

I'm trying to achieve algorithm that will extract text from few areas(marked with red color) on label(similar to attached image) and QR code on a single photo taken with mobile camera so label may be ...
1
vote
0answers
149 views

Installing NLTK using WHL file -

I have previously used WHL (wheel) files to install various Python packages. But, it seems there's no such file for NLTK. Any workaround for this please? https://pypi.org/project/nltk/ The problem ...