Questions tagged [text-classification]
For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.
288
questions
0
votes
0
answers
6
views
Is there any way to aquire image and video URLs from a large amount of facebook ads for research?
I have aquired a lot of URLs to some political ads from the Meta ad library API. As far as i can tell, there is no way to automate the aquisation of the image and video URLs from these ads. Is this ...
0
votes
0
answers
20
views
Multi-output Classification?
I have a dataset consisting of 1 X (Textual Data) and 5 different y (Topics of each text) and each y can take values from 0 to 5. I need to develop a deep learning model to take text and predict all y....
0
votes
0
answers
14
views
Subjectivity Classification with BERT and Word2Vec
I am new to NLP and working on a final-year project to classify if a sentence is written from objective or subjective point of view, using BERT with Word2Vec.
The datasets I found for this project are ...
0
votes
0
answers
3
views
How can we integrate zero shot classification with a supervised classifier?
We have two sets of labels that are known and unknown to the supervised classifier (SC).
We infer for a test example using the SC and a zero shot classifier (ZC).
Let's assume, our inference datapoint ...
0
votes
0
answers
8
views
Constraining Latent Dirichlet Allocation to a single topic per word
From the Wikipedia:
LDA assumes the following generative process for a corpus $D$
consisting of $M$ documents each of length $N_i$
Choose $ \theta_i \sim \operatorname{Dir}(\alpha) $, where $ i \in \{...
- 101
0
votes
0
answers
15
views
Benchmark dataset with both images and tabular data
In the medical sector, there are situations in which an image dataset is associated with a tabular dataset containing different features but the same labels as the image dataset. For example, suppose ...
- 221
1
vote
1
answer
10
views
Does high number of output labels affect the performance of BERT and how to handle the class imbalance issue while doing multi text classification?
I am using BERT to do multiclass text classification. The number of output classes I have to predict from is: 116 and there is high degree of class imbalance that I see.
We have the following kind of ...
0
votes
0
answers
5
views
Which is the best choice for evaluating models on small and unbalanced textual datasets?
We are dealing with a small multilabel dataset (around 15k samples) of texts that is imbalanced. Some classes have more than 4k samples and others have around 700 samples. We are using a classifier ...
- 139
1
vote
1
answer
30
views
Is it valid changing the classification treshold of neural networks for improving the classification performance?
I'm dealing with text classification using BERT pre-trained model with a multiclass imbalanced dataset.
When we use a 0.5 default classification threshold we obtain a f1 measure of around 0.7.
But we ...
- 139
0
votes
1
answer
22
views
using a feature that is only available during training
I'm working on a project that aims to classify JIRA issues into their relevant owner group.
An issue has the following text features:
Summary
Description
Comments
all of which are text based.
During ...
- 167
0
votes
0
answers
24
views
How do I improve the accuracy of a BERT-based multilabel text classification model?
I have a database table with 79,512 rows, each of which describes a category. Each row has a title and a description, and can even have a supercategory. Often, supercategories have categories.
I'm ...
- 149
0
votes
0
answers
26
views
Multi-label classification inference
I am working on a multi-label classification with transformers. I want to assign tags to input text. First, I have trained a model multiclass and with the pipeline function I can retrieve all possible ...
- 1
1
vote
1
answer
36
views
Topic classification on text data with no/few labels
I would like to achieve a classification of a text input into predefined categories.
From what I have understand unsupervised approach are unfeasible if my target label is something very rare in ...
- 45
0
votes
0
answers
14
views
How to deal with ambiguous classification outputs that exceed the specified threshold but are too close together?
I have a simple classification setup (intent classification). Once an input is received it's parsed using Multinomial Logistic Regression and then a score is predicted for each class. I pick the ...
0
votes
0
answers
98
views
How to deal with DataCollator and DataLoaders in Huggingface?
I have issues combining a DataLoader and DataCollator. The following code with DataCollatorWithPadding results in a ...
- 101
0
votes
0
answers
15
views
Can I use Deep Learning Algorithm between every two columns of excel with text data?
If my data has this format
Now can I use LSTM in between each column? I have to classify keyword into column B first and then Column A. are there any other ways I can look into? But I had larger data ...
- 21
0
votes
0
answers
10
views
Text Classification Model unable to learn
I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases.
I have researched for possible ...
- 1
0
votes
1
answer
18
views
Classify E-commerce URLs into predefined classes
How can I classify an E-commerce URL Page into the following categories,
Cart
Payment
Product Page
Checkout
How can I achieve this with the url and page title in my hand? I have tried multiple ways ...
- 41
1
vote
0
answers
10
views
Boosting the effect of some of the features in SVM
I'm doing text classification with SVM. I'm using Tfidf vectorization. In addition to the text vectors, I have a context data denoting the possible outcomes of the prediction. For example, I have a ...
- 121
3
votes
4
answers
989
views
How to automatically classify a sentence or text based on its context?
I have a database of sentences which are about different topics. I want to automatically classify each sentence with the one or more relevant tags based on the context of the sentence as shown below:
...
- 185
0
votes
0
answers
20
views
Tensorflow text classification with 'int' and 'tf-idf' vectorizer
My question is simple. I tried a text classification with both 'int' and 'tf-idf' vectorizers. I'd expect it would classify better with tf-idf but the scores are 0.65 for 'int' and 0.60 for 'tf-idf'. ...
- 121
0
votes
0
answers
7
views
Tensorflow text classification with subject for each text
I want to classify texts with additional input 'text' subject. I acquire these subjects from wikidata 'instance of' properties. I designed a neural net model as below. Network takes texts and subjects ...
- 121
0
votes
0
answers
24
views
Augmented text classification with Knowledge Graphs
There is a paper 'Learning beyond datasets: Knowledge Graph Augmented Neural
Networks for Natural language Processing'. In this paper a method for classifying texts with additional knowledge graph ...
- 121
0
votes
2
answers
47
views
NLP vs Keyword-Search. which one is the best?
I have constructed a natural language processing (NLP) model with the aim of identifying technology keywords within text. The model is trained on a large dataset that contains over 400,000 phrases and ...
- 103
0
votes
1
answer
67
views
LSTM model is producing really bad results for multiclass text classification for imbalanced small dataset
I am training a LSTM model on my current dataset to predict the multiclass categories - there are 18 mutually exclusive categories and the dataset has ...
- 163
0
votes
1
answer
15
views
Sequence labels with prior knowledge about valid labels
I am working on a text classification problem where on inference, there is a known set of valid classifications which is smaller than the set of all possible labels.
Example: There are 5 labels: L1, ...
- 103
1
vote
1
answer
70
views
What is the right processing order when working with a dataset that already consists of test and train data?
I want to work on the following task:
Text Classification using Deep Learning models and a Transfer Learning model.
The notebook that I'm creating should include the following steps:
Data ...
- 35
0
votes
0
answers
4
views
Visualizing data distribution of categorized texts
I'm trying to develop an SVM model to classify some texts into positive and negative. For the training phase, I'm going to use a dataset containing 2400 comments from Twitter, classified into positive ...
- 1
0
votes
0
answers
16
views
Text Annotation
How to annotate aspect-based sentiment analysis using docanno? I want to annotate data like the given piture
0
votes
0
answers
65
views
How to find the optimal number of samples for fine-tuning a pre-trained language model for text classification?
I'm trying to fine-tune a pre-trained language model (PLM) for text classification. The dataset that I'm using for fine-tuning includes about 40k samples.
I wonder if I should use the whole dataset or ...
- 115
1
vote
0
answers
24
views
NLP - F1 score for positive class drops to 0 after data augmentation
I'm working on a 3-class text classification problem where my initial class distribution looked like this:
positive: 50%
negative: 25% and
neutral: 25%
And training on a model on this slightly ...
- 215
1
vote
1
answer
18
views
Handling missing value in column having textual data
I have been working on a supervised ML use case where dataset has Numerical (Price), Categorical(Category) and Textual data(Description) as features. Description feature has about 30% missing values. ...
0
votes
0
answers
46
views
improve accuracy for LinearSVC (multiclass text classification)
I'm working on a project in which I'm trying to classify bugs (taken from Jira) to their relevant assignee group.
After creating and cleaning the dataset (~50000 records), my best results have always ...
- 167
0
votes
0
answers
71
views
CNN-BERT Text Classification good results on train and val, but bad prediction on testing
I built a Keras model to predict hoax news and true news using the CNN-BERT Text Classification algorithm with Categorical Classification, with label 1 indicating a hoax and 0 indicating true news.
...
- 1
0
votes
0
answers
17
views
What input does SVM consider when doing the text classification?
I was using SVM for text classification
pipe_lr1 = Pipeline(steps=[('cv',TfidfVectorizer()),
('lr_multi',MultiOutputClassifier(LinearSVC()))])
...
- 21
1
vote
1
answer
63
views
Do Sampling before or after TFIDF step?
This is a multiclass text classification problem. The dataset has a class imbalance and I'm planning to use a sampling technique before modeling.
Should the sampling be done before/after the ...
- 394
0
votes
1
answer
15
views
Ordering training text data by length
If I have text data where the length of documents greatly varies and I'd like to use it for training where I use batching, there is a great chance that long strings will be mixed with short strings ...
1
vote
0
answers
17
views
How to design and use CNNs for sentence classification?
I'm playing around with using CNNs for sentence classification. Basically all models I found implement the same model proposed in Convolutional Neural Networks for Sentence Classification (Kim, 2014), ...
- 171
1
vote
0
answers
31
views
Text + tabular data for classification
I have a dataset of "mixed" types e.g
text | date | amount | supplier
-----+--------+---------+------------
where ...
- 440
0
votes
2
answers
93
views
Accuracy decreases after adding more samples
I'm working on a multiclass text classification task (5 classes).
I've 2 types of datasets:
regular (~22000 samples)
dataset of duplicates (~19000 samples)
I've written a logic that labels them all.
...
- 167
0
votes
0
answers
13
views
Is there a way to choose the most appropriate option from mapping file while predicting the target label?
I have a multi class classification problem where I am predicting a target label using two error text fields.
Input data looks like this:
SNo.
Error Description
Error Trace
Target Label
1
some text
...
0
votes
0
answers
6
views
Textrank to Extract importance sentences
I used text rank to extract the most important sentences from a set of tasks that some professions perform, but I don't know if it would be correct to just sum the textrank outputs to know the output ...
3
votes
4
answers
1k
views
Accuracy is getting worse after text pre processing
I'm working a multi-class text classification project.
After splitting the dataset into train and test datasets, I've applied the below function on the train dataset (AKA pre processing):
...
- 167
0
votes
0
answers
186
views
Pre-trained BERT model running slow on CPU
I'm running a inference model using a pre-trained BERT model (BERTikal). The model works but is not fast enought running on CPU. It's taking about 5 minuts to classify a batch of 300 phrases.
The the ...
2
votes
1
answer
22
views
Newbie questions: real-time clustering of messages
I'm very much a newbie in NLP, so please accept my apologies if this is an obvious question, the wrong place to ask it or any other error I could be making.
I am considering using NLP for some subset ...
- 121
0
votes
0
answers
16
views
Predicting whether or not text is of a specified topic (topic defined by key words and phrases)
I was looking into binary classification methods for classifying whether or not a given text is related to a topic that is defined by given key words and phrases (e.g. the topic is meals and key ...
0
votes
0
answers
34
views
Is it possible to use text Auto Encoders without text generation?
I have a use case where I have large texts, and a lot of it. Pretty often the sequence length exceeds 1000 tokens. I need a lower dimensional compression of the texts as an input for a classifier. The ...
- 101
1
vote
1
answer
69
views
Handling of readability scores for short texts
I have a classification problem using emails as my dataset.
I would like to use scores from various readability formulas as features for the classification. However, most of them are defined for ...
- 518
1
vote
0
answers
59
views
How do I use a machine learning or AI algorithm on this kind of data?
I have a text data set with keyword, thematic, Sub thematic, intent Now I have to train a keyword then first classify into sub thematic, then into thematic and then into intent. the data looks like ...
- 11
0
votes
0
answers
23
views
Text mining technique for similarity index (account for spelling mistakes)
I only use Python
Context:
This is my first time ever doing this so I have no idea where to start.
I have a list of 20,000 words. It can contain emotes or different languages. I need to find a way to ...
- 113