Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
1
vote
2answers
20 views

Automatically finding business opportunities in text documents

I am new to machine learning and NLP. I am exploring the possibility of using one of these approaches to automatically examine a large collection of text documents and determine, first of all, if they ...
0
votes
1answer
25 views

Text clustering model on small dataset

Is there any way to run any clustering model on a small dataset with 290 text records (minimum character size is 100)?
0
votes
0answers
10 views

Keyword Imputation Using Machine Learning and NLP

I am fairly new to machine learning and data imputation so forgive me if I am using wrong words or not feasible ideas. I have a table as follows, with text in the Headline column and a list of ...
1
vote
0answers
7 views

I'm looking for some scripts or tools that can take some text, understand and generate multichoice questions, cloze deletion, pop quiz [closed]

I'm a fairly entry level coder in python but I was hoping for some guidance on if, or how I could load a block of text to generate random questions to test comprehension. My goal is to conduct a large ...
1
vote
0answers
44 views

How to extract numerical information from text descriptions

I have an attribute that is the description of an operation (i.e description of a building consent), I need to translate this to a mathematical operation. I need to find out the new number of dwelling ...
0
votes
0answers
21 views

NLP conversation data - Pre-processing steps

I have text data, so data that has been transcribed from conversations from employees to customers. So each call has a recording that has been transcribed to text. I am looking to do some analytics ...
2
votes
0answers
22 views

Unstructured data not template based to structured data

I am working on a project where my goal is to extract data from a large diversity of pdf that does not follow a template (i.e unstructured data not template based). My ORC part works well and now I am ...
0
votes
0answers
20 views

Problem Direction - Text Data - Conversation Classification

The problem I have a problem where I have text data that has been transcribed from a conversation. These conversations have been marked as a pass or fail in terms of compliance by a person, ie they ...
1
vote
1answer
21 views

Methods for finding characteristic words for a group of documents in comparison to another group of documents?

I'm working on a problem of anomaly detection, where at the end of the anomaly detection I will have a group of documents consisting of a title of each object that was flagged as anomalous. At the ...
2
votes
2answers
39 views

How to stop a text-classification model from depending on only couple of the words from input text instead of entire sentence?

I have a text classification deep-learning model, which takes in a text and outputs a softmax probability. I am using glove embeddings to represent my input text in numerical form for the DL model. ...
2
votes
1answer
37 views

Clustering Strings Based on Similar Word Sequences

In my dataset I have a feature having below data : Input Feature Brain Dementia Routine(Comfortone) Morning Check Dementia Brain-Routine(Comfortone) Brain MRA Routine (Comfortone) Brain-Dementia/...
0
votes
2answers
63 views
0
votes
1answer
25 views

how to filter out and discard irrelevant tweets in simplest way possible

I have lot of tweets and from which i need to filter out and discard irrelevant tweets. the criteria for a tweet to be irrelevant is very simple. if all that a tweet has is emojis or a single hastag ...
0
votes
2answers
33 views

How to stay up to date in NLP and use the best approaches?

There are many fast advancements in NLP field, BERT, RoBERTa, ALBERT, and XLNe, and no one can check the news or papers daily. Is there any way or site that keeps track of all these new developments ...
0
votes
0answers
7 views

How to work out the optimum threshold for BIRCH clustering

I am working with a large dataset so I thought that BIRCH would be an ideal clustering algorithm to apply. Can anyone suggest how I can work out what the optimum threshold would be?
1
vote
0answers
14 views

Choice of the number of topics (clusters) in textual data

I have a social science background and I'm doing a text mining project. I'm looking for advice about the choice of the number of topics/clusters when analyzing textual data. In particular, I'm ...
0
votes
2answers
42 views

Text mining match in Python [closed]

I have one column called A and one column called B where column B is a more detailed ...
2
votes
2answers
40 views

Why are words represented by frequency counts before embedding?

Before getting vector representations of words by embedding, the words are mapped to numbers. These numbers are chosen to be the frequency of that word in the dataset. Why does this convention exist? ...
2
votes
0answers
17 views

Who is supposed to label my sentiment analysis? Linguistics or psychologist?

I'm starting off my undergraduate research on text classification even though I'm still considered new to this topic. I've collected more than 20K data from Twitter. I've been trying to label the data ...
1
vote
1answer
30 views

Can I use a pre-trained model for sentiment analysis/text classification of unlabelled data?

I'm planning on working on a project where I'll have a large collection of tweets about coronavirus vaccines. None of the tweets will have a label (e.g. positive, neutral, negative). Therefore I won't ...
0
votes
0answers
14 views

Restrict Date parser in certain cases

Sorry if the title wasn't self-explanatory. Here is a detailed version. I created a data parser to parse dates from resumes. The ultimate goal is to find how many years of work experience a candidate ...
1
vote
1answer
19 views

How is the connection between Text Mining, NLP and Tasks like Tokenization, Lemmatization, Stop-word Removal etc.?

I am new to the whole world around Big Data and Text Mining. It took me a while to understand all the connections and to be able to classify the buzzwords. But there's one thing I still don't ...
0
votes
0answers
18 views

Using multiple TF-IDF to create a feature

I have around 200k comments and I extracted the top 200 words (without stop words) out of their content. Each comment is linked to a specific date. I would like to ask a very stupid question: Is it ...
1
vote
1answer
37 views

Document ranking on a web scraped dataset without any labelled data

I want to create a document ranking model which returns similar rows in the dataset for a sample query. The text in this corpus is standard english but without any labels (ie no query-related ...
1
vote
0answers
44 views

How to extract "defined skills" from texts documents? [closed]

I have a question with automatic natural language processing, I would like to automatically visualize skills (communication, marketing, statistics, etc.) taken from a document. I already have a ...
0
votes
0answers
25 views

NLP for recognising abstract concepts

I know that NLP algorithms can be trained to recognise the topic of a document or part of a document. I would like to train an algorithm to recognise certain abstract concepts. For example I want to ...
1
vote
1answer
41 views

what can be done using NLP for a small sentence samples?

I am new to NLP. I have few 100 textual sentences (100 rows in dataframe) with an average word length of 10 in a sentence. I would like to know what interesting insights (simple descriptive to ...
1
vote
0answers
39 views

Is it possible to combine cnn and rnn?

I would like to know if it is possible to combine rnn and cnn. I explain you : I have pictures of bikes, cars and moto and every pictures is linked to a text. For instance for a car I can have the ...
1
vote
1answer
40 views

How can i extract words from a single concatenated word?

I'm stuck on this problem and would love some input. I have mulitple words such as getExtention, getPath, someWord or someword and i want to separate each concatinated words into its own words such as:...
1
vote
0answers
46 views

Measuring Topic-coherence score & optimal number of topics in LDA Topic Modeling in Orange data mining

I'm trying to build an Orange workflow to perform LDA topic modeling for analyzing a text corpus (.CSV dataset). Unfortunately, the LDA widget in Orange lacks for advanced settings when comparing it ...
2
votes
1answer
45 views

Should bag of words in training set include test set data when doing text classification?

I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
0
votes
0answers
13 views

Deep learning detect reference boundary in text (or number of references in text)

I have several documents that either contain or don't an X number of references. I would like to build a model that can detect the number of references if any in a text. I've been thinking for ...
0
votes
0answers
16 views

Simple Question: How to Load corpora (.txt files) in R

Context: I have 47 .txt files which are all technically small corpora. They have been lowercased and full-stops have been removed. Question: Would anyone know how to or be able to point me to a ...
0
votes
0answers
25 views

How to extract important text markers from samples to identify patterns?

Problem I have collected a decently large set of movie trailer titles from various Youtube trailer channels. I'd like to extract or infer the movie title and release year from this set in some way to ...
2
votes
1answer
35 views

How do I split contents in a text that would include two or more different themes (context) in NLP?

For example, a text: "The airlines have affected by Corona since march 2020 a crime has been detected in Noia village this morning" the output should be: The airline companies have affected ...
0
votes
0answers
10 views

GCloud AI Platform for an ETL process is the right approach?

I'm involved in a project consisting in: Extracting from the internet a document (file of 8/10 pages) Processing it (extract text, clean, tokenize and lemmatizing) Loading it to a Relational data ...
2
votes
0answers
37 views

Social media text analysis

I'm currently analyzing Korean social media text. The below are the steps of the analysis. Collect/crawling text data from social media (e.g. Twitter, Facebook), which are related to specific topics. ...
0
votes
1answer
18 views

What is the common practice for NLP or text mining for non-English?

A lot of natural language processing tools are pre-trained with corpus in English. What if ones need to analyze, say, Dutch text? The blogs I find online are mostly saying traslating text into English ...
0
votes
1answer
21 views

Extracting and tagging/labeling text from pdfs [closed]

I'm trying to extract paragraphs from 230 pdfs that have similar formats, and then label the paragraphs based on the pdf's section headings. Normally, I'd convert the pdf to txt files or extract the ...
0
votes
2answers
53 views

Determining whether a sentence is "cliche" using NLP

I have a collection of essays from students. Each essay is about the same topic and of the same word length. My goal is to develop a machine learning algorithm that pinpoints "cliche" ...
0
votes
0answers
200 views

How to measure the Optimal-number of Topics & Topic-Coherence score in Orange's LDA Topic-Modeling?

I'm trying to build a suitable "code-free" Topic Model in "Orange Data Mining" software for the experimental part of my Thesis, in order to analyze the themes and trends of ...
0
votes
1answer
101 views

Extracting Products Name from Unstructured text

I have unstructured text like this ...
0
votes
1answer
28 views

How to identify the feature that make the model misclassifed in text classification

Hi I am working on social media financial THAI text classification, the problem with this one is the confused classes, the misclassified prediction has a pattern that consistent as a pair. and I want ...
0
votes
0answers
10 views

outdoor/indoor recognition in text

For a project I should train a model that predicts whether a certain text given as input (e.g the chapter of a book) is set indoor or outdoor. I do not have any labelled data. Could you point to ...
0
votes
0answers
27 views

URL classification problem with small training set

I have a set of a few thousand organisations (org) with some attributes (name, location, and type) and I want to identify the official URL for each organisation from a set of 10 URLs retrieved from ...
1
vote
1answer
137 views

Extracting Names using NER | Spacy

I'm new to NER and I've been trying to extract names using Spacy. Here's my code: ...
1
vote
1answer
17 views

How many words should be taken as features in a ML problem?

I would like to ask you how many words should be taken as features in a ML program. For example, if I have 30000 distinct words to make a vocabulary, what would a good number be? I am currently ...
0
votes
1answer
25 views

Extracting structured data from semi structured data

I want to use machine learning and NLP to convert semi-structured data in text files to structured data by predicting the patterns in the files and splitting the fields for example if I have a text ...
0
votes
0answers
15 views

Analysis of table to make compressing model

I have a table with hundred of entries in this style: //HEADER//CODE1//decimals_here//CODE4//decimals_here//CODE3//decimals_here//CODE5//decimals_here//ENDING// I ...
2
votes
1answer
32 views

Extracting events with attributes from unstructured text

I am scraping websites of organisations (mostly retailers) and I want to use NLP to extract information from the websites’ unstructured text. The first thing I want to do is to identify covid-related ...

1
2 3 4 5
12