Questions tagged [text-mining]

Refers to a subset of data mining concerned with extracting information from data in the form of text by recognizing patterns. The goal of text mining is often to classify a given document into one of a number of categories in an automatic way, and to improve this performance dynamically, making it an example of machine learning. One example of this type of text mining are spam filters used for email.

Filter by
Sorted by
Tagged with
1 vote
1 answer
8 views

Building a graph out of a large text corpus

I'm given a large amount of documents upon which I should perform various kinds of analysis. Since the documents are to be used as a foundation of a final product, I thought about building a graph out ...
user avatar
0 votes
1 answer
19 views

How to deal with one output for multiple inputs?

Hei! I want to train a model, that predicts the sentiment of news headlines. I've got multiple unordered news headlines per day, but one sentiment score. What is a convenient solution to overcome the ...
user avatar
0 votes
0 answers
9 views

Read corpus from a csv file in Orange3

I have twitter text as an Excel file: every line is one one tweet. How do I view this corpus in Orange3? I don't understand why I can't simply see this corpus. As you can see in the image below, the ...
user avatar
0 votes
0 answers
6 views

Atomic tasks from a complex task using NLP

I have a problem statement when I need to find all the tasks that the server had to do based on a complex task. Example, in a 3D modeling scenario, if the model is queried with a complex task such as &...
user avatar
  • 41
0 votes
0 answers
12 views

Extract a numeric attribute from partially unstructured text for each word of a vocabulary

Given a vocabulary v = {'sales', 'units', 'parts', 'operators', 'revenue'} and strings such as ...
user avatar
3 votes
1 answer
24 views

NLP text representation techniques that preserve word order in sentence?

I see people are talking mostly about bag-of-words, td-idf and word embeddings. But these are at word levels. BoW and tf-idf fail to represent word orders, and word embeddings are not meant to ...
user avatar
  • 359
0 votes
0 answers
20 views

How to convert a string variable containing comments to a variable with integers to be used in neural networks?

I am working with data contains comment variable like imdb data. ...
user avatar
0 votes
0 answers
13 views

Deep Regression Ensembles(DRE) - text analysis

I read an article about Deep Regression Ensembles(DRE), which can outperform DNN using SDG. My question is could I use DRE in text classification? (for example, I can use it instead of LDA) What about ...
user avatar
1 vote
1 answer
18 views

How do you handle the free-text fields in tabular data in ML/DL?

While we see a number of cases where the input data is only a single text fields (for the X variable) in NLP tasks, e.g. a tweet with a sentiment label being the only numerical field. But how do you ...
user avatar
  • 359
2 votes
1 answer
28 views

Measuring The Discriminating Power of Clustering

I am measuring the "proximity" of certain words (proper nouns) within a certain text (Silmarillion) by determining the occurrences of these words within the text and, via binning, creating a ...
user avatar
  • 121
0 votes
0 answers
6 views

Removing footers from text scraped from news sites

I am wondering if anyone knows of libraries out there that will remove footer material from text scraped from news sites, like the material in italics below: “Go, I’ll catch up with you,” the woman ...
user avatar
  • 101
0 votes
0 answers
10 views

Orange Document Map

I've used the Document Map widget in Orange to identify all countries mentioned in the corpus. There appears to be a bug in the document map: On the map itself some countries are not colored (high ...
user avatar
  • 1
3 votes
2 answers
40 views

How to do sentence segmentation without loosing sentence's subject?

I have some text with different lengths, I want to split it into separate clauses but I also want to preserve the subject For example; ...
user avatar
0 votes
1 answer
175 views

Should we clean text data before applying Vader for getting sentiment

What I meant by data cleaning is that Removing Punctuations Lower Casing Removing Stop words Removing irrelevant symbols, links and emojis According to my knowledge, things like Punctuations, ...
user avatar
0 votes
0 answers
6 views

What to use for relationship extraction

Is there a simple way or a specific library for relationship extraction? I know I can just parse the sentence using spacy, but I'll end up with a different relationship for every sentence. What I'm ...
user avatar
  • 101
0 votes
0 answers
21 views

NLP: Finding questions or information sought in a paragraph

I am a newbie in Natural Language Processing. I recently explored IBM Natural Language Understanding (NLU) based on the tutorials given on their website. But still I am unable to proceed toward my ...
user avatar
  • 101
0 votes
0 answers
21 views

Unsupervised clustering for Text labelling

I have millions semi structured text descriptions of a job requirement. Which needs to be labelled, such as the number of hours, years of experience required, shifts, certifications, licensure etc., ...
user avatar
0 votes
0 answers
65 views

How to get fine-grained sentiment score from text data under unsupervised learning?

In my experience I have only used LSTM models to do sentiment classification tasks on text data under supervised learning. For example, the imdb dataset from keras which is a binary classification ...
user avatar
0 votes
1 answer
56 views

extracting data from unstructured pdfs

I have about 200,000 PDFs made up of 20 different designs. i.e In an organization, different (20) departments issue monthly award submission requirements. Each department has its own document format. ...
user avatar
  • 103
2 votes
2 answers
94 views

How to improve language model ex: BERT on unseen text in training?

I am using pre-trained language model for binary classification. I fine-tune the model by training on data my downstream task. The results are good almost 98% F-measure. However, when I remove a ...
user avatar
  • 123
1 vote
1 answer
94 views

How to evaluate triple extraction in NLP?

I am current NLP work, I am extracting triples using triple extraction function in Stanford NLP and Spacy libraries. I am looking for a good method to evaluate how good the extraction has been? Any ...
user avatar
  • 133
0 votes
0 answers
6 views

How to Analyze Specificity (Opposite to Vagueness) of Textual Data

I am quite new to text mining approaches and What I currently know and have used include sentiment analysis, topic modeling, and some applications of classifications using text data sets. I was ...
user avatar
  • 123
1 vote
2 answers
25 views

Automatically finding business opportunities in text documents

I am new to machine learning and NLP. I am exploring the possibility of using one of these approaches to automatically examine a large collection of text documents and determine, first of all, if they ...
user avatar
  • 11
0 votes
1 answer
36 views

Text clustering model on small dataset

Is there any way to run any clustering model on a small dataset with 290 text records (minimum character size is 100)?
user avatar
  • 129
0 votes
0 answers
22 views

Keyword Imputation Using Machine Learning and NLP

I am fairly new to machine learning and data imputation so forgive me if I am using wrong words or not feasible ideas. I have a table as follows, with text in the Headline column and a list of ...
user avatar
1 vote
0 answers
12 views

I'm looking for some scripts or tools that can take some text, understand and generate multichoice questions, cloze deletion, pop quiz [closed]

I'm a fairly entry level coder in python but I was hoping for some guidance on if, or how I could load a block of text to generate random questions to test comprehension. My goal is to conduct a large ...
user avatar
1 vote
0 answers
50 views

How to extract numerical information from text descriptions

I have an attribute that is the description of an operation (i.e description of a building consent), I need to translate this to a mathematical operation. I need to find out the new number of dwelling ...
user avatar
0 votes
0 answers
42 views

NLP conversation data - Pre-processing steps

I have text data, so data that has been transcribed from conversations from employees to customers. So each call has a recording that has been transcribed to text. I am looking to do some analytics ...
user avatar
1 vote
0 answers
27 views

Unstructured data not template based to structured data

I am working on a project where my goal is to extract data from a large diversity of pdf that does not follow a template (i.e unstructured data not template based). My ORC part works well and now I am ...
user avatar
0 votes
0 answers
22 views

Problem Direction - Text Data - Conversation Classification

The problem I have a problem where I have text data that has been transcribed from a conversation. These conversations have been marked as a pass or fail in terms of compliance by a person, ie they ...
user avatar
1 vote
1 answer
24 views

Methods for finding characteristic words for a group of documents in comparison to another group of documents?

I'm working on a problem of anomaly detection, where at the end of the anomaly detection I will have a group of documents consisting of a title of each object that was flagged as anomalous. At the ...
user avatar
  • 133
2 votes
2 answers
41 views

How to stop a text-classification model from depending on only couple of the words from input text instead of entire sentence?

I have a text classification deep-learning model, which takes in a text and outputs a softmax probability. I am using glove embeddings to represent my input text in numerical form for the DL model. ...
user avatar
2 votes
1 answer
91 views

Clustering Strings Based on Similar Word Sequences

In my dataset I have a feature having below data : Input Feature Brain Dementia Routine(Comfortone) Morning Check Dementia Brain-Routine(Comfortone) Brain MRA Routine (Comfortone) Brain-Dementia/...
user avatar
0 votes
2 answers
466 views

calibrated classifier ValueError: could not convert string to float

Dataframe: ...
user avatar
  • 129
0 votes
1 answer
38 views

how to filter out and discard irrelevant tweets in simplest way possible

I have lot of tweets and from which i need to filter out and discard irrelevant tweets. the criteria for a tweet to be irrelevant is very simple. if all that a tweet has is emojis or a single hastag ...
user avatar
0 votes
2 answers
65 views

How to stay up to date in NLP and use the best approaches?

There are many fast advancements in NLP field, BERT, RoBERTa, ALBERT, and XLNe, and no one can check the news or papers daily. Is there any way or site that keeps track of all these new developments ...
user avatar
0 votes
0 answers
14 views

How to work out the optimum threshold for BIRCH clustering

I am working with a large dataset so I thought that BIRCH would be an ideal clustering algorithm to apply. Can anyone suggest how I can work out what the optimum threshold would be?
user avatar
  • 25
1 vote
1 answer
23 views

Choice of the number of topics (clusters) in textual data

I have a social science background and I'm doing a text mining project. I'm looking for advice about the choice of the number of topics/clusters when analyzing textual data. In particular, I'm ...
user avatar
  • 11
0 votes
2 answers
45 views

Text mining match in Python [closed]

I have one column called A and one column called B where column B is a more detailed ...
user avatar
2 votes
2 answers
49 views

Why are words represented by frequency counts before embedding?

Before getting vector representations of words by embedding, the words are mapped to numbers. These numbers are chosen to be the frequency of that word in the dataset. Why does this convention exist? ...
user avatar
  • 23
3 votes
1 answer
29 views

Who is supposed to label my sentiment analysis? Linguistics or psychologist?

I'm starting off my undergraduate research on text classification even though I'm still considered new to this topic. I've collected more than 20K data from Twitter. I've been trying to label the data ...
user avatar
  • 33
2 votes
1 answer
54 views

Can I use a pre-trained model for sentiment analysis/text classification of unlabelled data?

I'm planning on working on a project where I'll have a large collection of tweets about coronavirus vaccines. None of the tweets will have a label (e.g. positive, neutral, negative). Therefore I won't ...
user avatar
  • 131
0 votes
0 answers
17 views

Restrict Date parser in certain cases

Sorry if the title wasn't self-explanatory. Here is a detailed version. I created a data parser to parse dates from resumes. The ultimate goal is to find how many years of work experience a candidate ...
user avatar
1 vote
1 answer
22 views

How is the connection between Text Mining, NLP and Tasks like Tokenization, Lemmatization, Stop-word Removal etc.?

I am new to the whole world around Big Data and Text Mining. It took me a while to understand all the connections and to be able to classify the buzzwords. But there's one thing I still don't ...
user avatar
  • 27
0 votes
0 answers
21 views

Using multiple TF-IDF to create a feature

I have around 200k comments and I extracted the top 200 words (without stop words) out of their content. Each comment is linked to a specific date. I would like to ask a very stupid question: Is it ...
user avatar
1 vote
1 answer
56 views

Document ranking on a web scraped dataset without any labelled data

I want to create a document ranking model which returns similar rows in the dataset for a sample query. The text in this corpus is standard english but without any labels (ie no query-related ...
user avatar
  • 11
0 votes
0 answers
30 views

NLP for recognising abstract concepts

I know that NLP algorithms can be trained to recognise the topic of a document or part of a document. I would like to train an algorithm to recognise certain abstract concepts. For example I want to ...
user avatar
  • 101
1 vote
1 answer
64 views

what can be done using NLP for a small sentence samples?

I am new to NLP. I have few 100 textual sentences (100 rows in dataframe) with an average word length of 10 in a sentence. I would like to know what interesting insights (simple descriptive to ...
user avatar
  • 2,129
1 vote
0 answers
53 views

Is it possible to combine cnn and rnn?

I would like to know if it is possible to combine rnn and cnn. I explain you : I have pictures of bikes, cars and moto and every pictures is linked to a text. For instance for a car I can have the ...
user avatar
  • 11
1 vote
1 answer
49 views

How can i extract words from a single concatenated word?

I'm stuck on this problem and would love some input. I have mulitple words such as getExtention, getPath, someWord or someword and i want to separate each concatinated words into its own words such as:...
user avatar

1
2 3 4 5
12