Questions tagged [text]

Text is a type of data often used in data science projects involving natural language processing.

Filter by
Sorted by
Tagged with
0
votes
1answer
22 views

Keywords extraction for business rule text classification

I would like to classify texts without using any ML model. My idea is to find a list of keywords that I would assign to each class. Then when I need to classify a new text, I can compare it with my ...
1
vote
1answer
115 views

Why not rule-based semantic role labelling?

I have recently found some interest in automatic semantic role labelling. Most introductory texts (e.g. Jurafsky and Martin, 2008) present approaches based on supervised machine learning, often using ...
0
votes
0answers
5 views

How to generate syntactically correct text for CRNN-CTC text model?

Disregarding the image creation and labeling details, is there a way to generate syntactically correct text examples? As of my current understanding of the CTC model, it takes into consideration the ...
0
votes
0answers
25 views

Text classification length

I have a set of text examples I need to learn as class A, and they are of varying lengths, say 10 sentences to 1 sentence long. I have to parse a document to find those strings of text that match one ...
0
votes
0answers
25 views

Fake News Detection Classifier approach

I have the dataset related to any domain like sports, entertainment, politics, etc. I just want to know that the approach I am using for fake news detection is valid or not. As I do not want to use ...
0
votes
0answers
12 views

Handwritten Text Recognition with different char set

I was trying to understand how Handwritten Text Recognition works but here I am. I did a lot of research but still, I couldn't exactly understand how will HTR architectures work even with different ...
0
votes
0answers
29 views

NLP conversation data - Pre-processing steps

I have text data, so data that has been transcribed from conversations from employees to customers. So each call has a recording that has been transcribed to text. I am looking to do some analytics ...
5
votes
1answer
78 views

How to evaluate the similarity of two columns containing strings?

I am new to text processing and stuck on a problem to identify the similarity of columns. To detail the problem, consider we have two columns with string values: ...
0
votes
0answers
20 views

Problem Direction - Text Data - Conversation Classification

The problem I have a problem where I have text data that has been transcribed from a conversation. These conversations have been marked as a pass or fail in terms of compliance by a person, ie they ...
0
votes
0answers
28 views

How do I work with noisy real world text data for text classification?

I have a topic classification model built upon Bert, when I deploy my model people input strings of a random nature like : "aaaaaa" "aaa bbb" "ab ab ab" and so on. My ...
0
votes
1answer
35 views

Working with three types of data: numeric (integer, floats), images, and text for prediction

So I have three types of data (in title) and am wondering how I can combine the data. The target is numeric (price). My idea is to perform feature extraction on both the images and text, which would ...
1
vote
1answer
34 views

Topic modelling on long documents: intra document clustering first

I have a collection (around 1000) of very noisy, similar documents, that are each very long (>10 pages - 600 paragraphs) with multiple subsections - I want to perform topic modelling across the ...
2
votes
0answers
22 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
2
votes
1answer
429 views

How to impute missing text data?

Lets say I have a dataframe consisting of two text columns. By text, I mean the values in those columns are either sentences/paragraphs. In such a case, how do I handle missing 'NaN' values? If it ...
1
vote
1answer
17 views

How to find the probability of a word to belong to a dataset of text

I have two text datasets, one with people that have a certain medical condition and another of random patients. I want to figure out what words are more likely to show up in the dataset with that ...
1
vote
1answer
19 views

Performance measurement of an event extraction system

I have developed an event extraction system from text documents. It first clusters the data corpus and extracts answers for what, when and where questions. Final answers are determined by using a ...
0
votes
0answers
388 views

ValueError: Layer model expects 2 input(s), but it received 3 input tensors using generator

I am trying to fit a model using generator function and I get the following error: ...
4
votes
1answer
44 views

encoding of text data in NLP

I'm getting data using web scraping to create a dataset. I have a 'company' column that contains the names of the companies. I would like to encode this column but i don't know how to find the ...
0
votes
0answers
16 views

What is a good way to extract blocks of texts from documents in various formats?

I have lots of documentation in pdf, ppt, xls files, which contain tables with text, pictures, headlines etc. The goal is to extract blocks of texts where the information is continuous, so to say. The ...
1
vote
0answers
75 views

Cluster words into groups of similar meaning (synonyms)

How can words be clustered into groups of similar meaning (synonyms)? I started with pre-trained word embeddings (e.g., Google News), which is great, but not perfect - a limitation arises because the ...
0
votes
0answers
8 views

What techniques are there to train custom sentence classification models with reasonable memory footprint?

We are currently working on tasks that involve user-inputted data (e.g., question-answers, short-answer-grading), with a framework that will allow them to be improved through active learning. However, ...
0
votes
0answers
34 views

Effective way to find similarity between utterance(short text) and question(long text)

The challenge I have is a bunch of questions(long text) that are closely matching with an user utterance (short text). I have tried cosine similarity & Tf-iDF, BM25,Jaccard similarity, etc., but ...
0
votes
1answer
152 views

Advantages of CNN vs. LSTM for sequence data like text or log-files

When do you tend to use CNN rather than LSTM (or the other way round) in classification or generation tasks of sequential data like text or log-data? What are the reasons for the decision and what ...
0
votes
1answer
32 views

How to identify the feature that make the model misclassifed in text classification

Hi I am working on social media financial THAI text classification, the problem with this one is the confused classes, the misclassified prediction has a pattern that consistent as a pair. and I want ...
0
votes
0answers
21 views

text elaboration with neural network non natural language

I'm trying to understand if it's possible to elaborate with the neural network 2 input strings and predict a result string. I'm doing this job manually and it's really time consuming: basically i have ...
0
votes
0answers
23 views

Use Large Existing Dataset to Extract Information From Text

What I have: I have a large dataset of documents and their data. So I have the text of about 1M documents, and I know, for example the invoice number, of each one. What I need: Is there a way to use ...
0
votes
0answers
132 views

Fundamental difference between Tokenizer and TextVectorization in Keras

Is there any fundamental difference between Tokenizer at the word level and the TextVectorization in Keras?
0
votes
1answer
69 views

Word embedding for a single word

I want to tackle email similarity with a word embedding approach, not an expert in embedding text but it is possible to embed emails where similar emails have similar vectors? is that okay?. I know ...
0
votes
1answer
37 views

How to remove irrelevant text data from a large dataset

I am working on a ML project where data were coming from a social media, and the topic about the data should be depression under Covid-19. However, when I read some of the data retrieved, I noticed ...
3
votes
1answer
216 views

Extract date/duration from text

The text and output to be extracted are similar to the following : "Check it every two weeks" - two weeks "Check it on day 1 and day 14" - day 1 and day 14 "day 19 and day ...
0
votes
0answers
1k views

Excel error may be caused by pandas writing or big data? advise needed

I am reading multiple xml files extracting some data then forming a pandas Dataframe with my data. These are the main steps that I do: open an xml file extract some elements create a pandas dataframe ...
-1
votes
1answer
32 views

How to specify a location for text in a graph plotted by Python?

I would like to plot a graph of actual and predicted values with Python after doing a regression. I used the following codes. However, the text "R^2=0.91" is placed on the right hand side ...
1
vote
1answer
17 views

Which phrase should be returned in case of multiple matches when comparing text?

I want to compare one sentence to some other sentences using the Bag of Words model. Suppose that my comparing sentence is: I am playing football and there are three more sentences that I want to ...
1
vote
1answer
24 views

How to Identify Repeating Data Entries when the Repeated Entries are Spelled or Constructed Differently

I have a dataset of entries and a variable for the owner of the entry. Some of these people occur more than once. However, the names are sometimes written differently. I want to eventually be able to ...
1
vote
1answer
18 views

Model to detect specific semantic content without labeled data

I want to build a model that can detect sentences that discuss requests for communication - like 'email me', 'phone us', 'contact us', etc. However, I do not have any labeled data which I can use to ...
0
votes
1answer
38 views

Python to clean miswritten words with repetitive letters such as "wwwwooorrrrddss" to "words"

When cleaning text-data in Python3 for NLP, are there are any 'common practices' when it comes to addressing repetitive letters in words such as "wwwwooorrds" to "words", or "...
1
vote
1answer
61 views

distribution difference between image and text

Once for the task of image captioning I've read that, the features extracted from image and text by deep networks are from two different worlds and got different distribution. My question is how is ...
1
vote
1answer
44 views

What's the difference between sequence preprocessing and text preprocessing in Keras?

In Keras, we mainly have three types of preprocessing, i.e., sequence preprocessing, text preprocessing, and image preprocessing. However, for me, I think the meanings of the word "sequence" ...
0
votes
0answers
13 views

Converting a string to a recommendation type string

I am trying to build a recommendation system and some of the labels are ...
1
vote
1answer
612 views

Clustering Tweet Data using DBSCAN Algorithm

I am doing a tweet clustering using DBSCAN algorithm. I use all the preprocessing steps and convert sentences to vector format before applying the algorithm. ...
1
vote
0answers
21 views

NLP Classification labels have many similarirites, converge to and replace to only have one

I been trying to use the fuzzywuzzy library in Python to find the percentage similarity between strings in the labels. The problem I am having is that there is still many strings that are really ...
0
votes
3answers
42 views

Distinguish randomly generated texts from reasonable for human texts [closed]

I have strings short texts of 2 types: '23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ... and ...
1
vote
0answers
78 views

How can I get a value of context vector in GPT?

I'm a newbie in NLP and I'm now stuck in GPT. The question I'm struggling with is related to a term 'context vector' It says in the following (sorry that the material provided is written in korean) ...
1
vote
2answers
46 views

Chinese word segmentation using neural networks

Chinese text uses a character set containing tens of thousands of characters. Words in Chinese are most commonly made up of 1, 2 or 3 characters. There are no spaces or other markers between words in ...
0
votes
0answers
18 views

Classification of text articles found in different PDFs

I am trying to figure out the approach to connect articles present in different pages or PDFs. The historical data that I have are: Text Articles (broken down in headlines, body text, body headline ...
0
votes
2answers
128 views

Need for Dense layer in Text Classifcation

While creating a model for text classification, what is the need for a Dense Layer? I noticed in multiple examples the following is the structure. A softmax is what required right instead of the Dense ...
1
vote
1answer
40 views

how to classify text based on more than one column

I passages of text to classify by topic. I am using scikit learn, e.g. linear svc, but open to other options. Currently, use only the text of each passage (column labeled ...
2
votes
2answers
485 views

Clustering mixed data types - numeric, categorical, arrays, and text

I have a dataset with 4 types of data columns: ...
2
votes
1answer
243 views

Are there any open-source text annotation for multi label classification tools?

I have a large texts in each document and I want to know if there are any open source text annotation tools available online for multiple label annotation. Each sentence takes two labels. If there ...
1
vote
0answers
27 views

Paragraph extraction from text

I am trying to separate scanned pages of a 3 column book into paragraphs. On the pages there can be images located in an arbitrary location, occupying part of one, two or all three of the columns. ...