Questions tagged [text-classification]

For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

Is there any way to aquire image and video URLs from a large amount of facebook ads for research?

I have aquired a lot of URLs to some political ads from the Meta ad library API. As far as i can tell, there is no way to automate the aquisation of the image and video URLs from these ads. Is this ...
0 votes
0 answers
20 views

Multi-output Classification?

I have a dataset consisting of 1 X (Textual Data) and 5 different y (Topics of each text) and each y can take values from 0 to 5. I need to develop a deep learning model to take text and predict all y....
0 votes
0 answers
14 views

Subjectivity Classification with BERT and Word2Vec

I am new to NLP and working on a final-year project to classify if a sentence is written from objective or subjective point of view, using BERT with Word2Vec. The datasets I found for this project are ...
0 votes
0 answers
3 views

How can we integrate zero shot classification with a supervised classifier?

We have two sets of labels that are known and unknown to the supervised classifier (SC). We infer for a test example using the SC and a zero shot classifier (ZC). Let's assume, our inference datapoint ...
0 votes
0 answers
8 views

Constraining Latent Dirichlet Allocation to a single topic per word

From the Wikipedia: LDA assumes the following generative process for a corpus $D$ consisting of $M$ documents each of length $N_i$ Choose $ \theta_i \sim \operatorname{Dir}(\alpha) $, where $ i \in \{...
  • 101
0 votes
0 answers
15 views

Benchmark dataset with both images and tabular data

In the medical sector, there are situations in which an image dataset is associated with a tabular dataset containing different features but the same labels as the image dataset. For example, suppose ...
  • 221
1 vote
1 answer
10 views

Does high number of output labels affect the performance of BERT and how to handle the class imbalance issue while doing multi text classification?

I am using BERT to do multiclass text classification. The number of output classes I have to predict from is: 116 and there is high degree of class imbalance that I see. We have the following kind of ...
0 votes
0 answers
5 views

Which is the best choice for evaluating models on small and unbalanced textual datasets?

We are dealing with a small multilabel dataset (around 15k samples) of texts that is imbalanced. Some classes have more than 4k samples and others have around 700 samples. We are using a classifier ...
  • 139
1 vote
1 answer
30 views

Is it valid changing the classification treshold of neural networks for improving the classification performance?

I'm dealing with text classification using BERT pre-trained model with a multiclass imbalanced dataset. When we use a 0.5 default classification threshold we obtain a f1 measure of around 0.7. But we ...
  • 139
0 votes
1 answer
22 views

using a feature that is only available during training

I'm working on a project that aims to classify JIRA issues into their relevant owner group. An issue has the following text features: Summary Description Comments all of which are text based. During ...
  • 167
0 votes
0 answers
24 views

How do I improve the accuracy of a BERT-based multilabel text classification model?

I have a database table with 79,512 rows, each of which describes a category. Each row has a title and a description, and can even have a supercategory. Often, supercategories have categories. I'm ...
0 votes
0 answers
26 views

Multi-label classification inference

I am working on a multi-label classification with transformers. I want to assign tags to input text. First, I have trained a model multiclass and with the pipeline function I can retrieve all possible ...
1 vote
1 answer
36 views

Topic classification on text data with no/few labels

I would like to achieve a classification of a text input into predefined categories. From what I have understand unsupervised approach are unfeasible if my target label is something very rare in ...
  • 45
0 votes
0 answers
14 views

How to deal with ambiguous classification outputs that exceed the specified threshold but are too close together?

I have a simple classification setup (intent classification). Once an input is received it's parsed using Multinomial Logistic Regression and then a score is predicted for each class. I pick the ...
0 votes
0 answers
98 views

How to deal with DataCollator and DataLoaders in Huggingface?

I have issues combining a DataLoader and DataCollator. The following code with DataCollatorWithPadding results in a ...
  • 101
0 votes
0 answers
15 views

Can I use Deep Learning Algorithm between every two columns of excel with text data?

If my data has this format Now can I use LSTM in between each column? I have to classify keyword into column B first and then Column A. are there any other ways I can look into? But I had larger data ...
0 votes
0 answers
10 views

Text Classification Model unable to learn

I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases. I have researched for possible ...
0 votes
1 answer
18 views

Classify E-commerce URLs into predefined classes

How can I classify an E-commerce URL Page into the following categories, Cart Payment Product Page Checkout How can I achieve this with the url and page title in my hand? I have tried multiple ways ...
  • 41
1 vote
0 answers
10 views

Boosting the effect of some of the features in SVM

I'm doing text classification with SVM. I'm using Tfidf vectorization. In addition to the text vectors, I have a context data denoting the possible outcomes of the prediction. For example, I have a ...
3 votes
4 answers
989 views

How to automatically classify a sentence or text based on its context?

I have a database of sentences which are about different topics. I want to automatically classify each sentence with the one or more relevant tags based on the context of the sentence as shown below: ...
  • 185
0 votes
0 answers
20 views

Tensorflow text classification with 'int' and 'tf-idf' vectorizer

My question is simple. I tried a text classification with both 'int' and 'tf-idf' vectorizers. I'd expect it would classify better with tf-idf but the scores are 0.65 for 'int' and 0.60 for 'tf-idf'. ...
0 votes
0 answers
7 views

Tensorflow text classification with subject for each text

I want to classify texts with additional input 'text' subject. I acquire these subjects from wikidata 'instance of' properties. I designed a neural net model as below. Network takes texts and subjects ...
0 votes
0 answers
24 views

Augmented text classification with Knowledge Graphs

There is a paper 'Learning beyond datasets: Knowledge Graph Augmented Neural Networks for Natural language Processing'. In this paper a method for classifying texts with additional knowledge graph ...
0 votes
2 answers
47 views

NLP vs Keyword-Search. which one is the best?

I have constructed a natural language processing (NLP) model with the aim of identifying technology keywords within text. The model is trained on a large dataset that contains over 400,000 phrases and ...
0 votes
1 answer
67 views

LSTM model is producing really bad results for multiclass text classification for imbalanced small dataset

I am training a LSTM model on my current dataset to predict the multiclass categories - there are 18 mutually exclusive categories and the dataset has ...
0 votes
1 answer
15 views

Sequence labels with prior knowledge about valid labels

I am working on a text classification problem where on inference, there is a known set of valid classifications which is smaller than the set of all possible labels. Example: There are 5 labels: L1, ...
  • 103
1 vote
1 answer
70 views

What is the right processing order when working with a dataset that already consists of test and train data?

I want to work on the following task: Text Classification using Deep Learning models and a Transfer Learning model. The notebook that I'm creating should include the following steps: Data ...
  • 35
0 votes
0 answers
4 views

Visualizing data distribution of categorized texts

I'm trying to develop an SVM model to classify some texts into positive and negative. For the training phase, I'm going to use a dataset containing 2400 comments from Twitter, classified into positive ...
  • 1
0 votes
0 answers
16 views

Text Annotation

How to annotate aspect-based sentiment analysis using docanno? I want to annotate data like the given piture
0 votes
0 answers
65 views

How to find the optimal number of samples for fine-tuning a pre-trained language model for text classification?

I'm trying to fine-tune a pre-trained language model (PLM) for text classification. The dataset that I'm using for fine-tuning includes about 40k samples. I wonder if I should use the whole dataset or ...
1 vote
0 answers
24 views

NLP - F1 score for positive class drops to 0 after data augmentation

I'm working on a 3-class text classification problem where my initial class distribution looked like this: positive: 50% negative: 25% and neutral: 25% And training on a model on this slightly ...
1 vote
1 answer
18 views

Handling missing value in column having textual data

I have been working on a supervised ML use case where dataset has Numerical (Price), Categorical(Category) and Textual data(Description) as features. Description feature has about 30% missing values. ...
0 votes
0 answers
46 views

improve accuracy for LinearSVC (multiclass text classification)

I'm working on a project in which I'm trying to classify bugs (taken from Jira) to their relevant assignee group. After creating and cleaning the dataset (~50000 records), my best results have always ...
  • 167
0 votes
0 answers
71 views

CNN-BERT Text Classification good results on train and val, but bad prediction on testing

I built a Keras model to predict hoax news and true news using the CNN-BERT Text Classification algorithm with Categorical Classification, with label 1 indicating a hoax and 0 indicating true news. ...
0 votes
0 answers
17 views

What input does SVM consider when doing the text classification?

I was using SVM for text classification pipe_lr1 = Pipeline(steps=[('cv',TfidfVectorizer()), ('lr_multi',MultiOutputClassifier(LinearSVC()))]) ...
1 vote
1 answer
63 views

Do Sampling before or after TFIDF step?

This is a multiclass text classification problem. The dataset has a class imbalance and I'm planning to use a sampling technique before modeling. Should the sampling be done before/after the ...
0 votes
1 answer
15 views

Ordering training text data by length

If I have text data where the length of documents greatly varies and I'd like to use it for training where I use batching, there is a great chance that long strings will be mixed with short strings ...
1 vote
0 answers
17 views

How to design and use CNNs for sentence classification?

I'm playing around with using CNNs for sentence classification. Basically all models I found implement the same model proposed in Convolutional Neural Networks for Sentence Classification (Kim, 2014), ...
  • 171
1 vote
0 answers
31 views

Text + tabular data for classification

I have a dataset of "mixed" types e.g text | date | amount | supplier -----+--------+---------+------------ where ...
0 votes
2 answers
93 views

Accuracy decreases after adding more samples

I'm working on a multiclass text classification task (5 classes). I've 2 types of datasets: regular (~22000 samples) dataset of duplicates (~19000 samples) I've written a logic that labels them all. ...
  • 167
0 votes
0 answers
13 views

Is there a way to choose the most appropriate option from mapping file while predicting the target label?

I have a multi class classification problem where I am predicting a target label using two error text fields. Input data looks like this: SNo. Error Description Error Trace Target Label 1 some text ...
0 votes
0 answers
6 views

Textrank to Extract importance sentences

I used text rank to extract the most important sentences from a set of tasks that some professions perform, but I don't know if it would be correct to just sum the textrank outputs to know the output ...
3 votes
4 answers
1k views

Accuracy is getting worse after text pre processing

I'm working a multi-class text classification project. After splitting the dataset into train and test datasets, I've applied the below function on the train dataset (AKA pre processing): ...
  • 167
0 votes
0 answers
186 views

Pre-trained BERT model running slow on CPU

I'm running a inference model using a pre-trained BERT model (BERTikal). The model works but is not fast enought running on CPU. It's taking about 5 minuts to classify a batch of 300 phrases. The the ...
2 votes
1 answer
22 views

Newbie questions: real-time clustering of messages

I'm very much a newbie in NLP, so please accept my apologies if this is an obvious question, the wrong place to ask it or any other error I could be making. I am considering using NLP for some subset ...
  • 121
0 votes
0 answers
16 views

Predicting whether or not text is of a specified topic (topic defined by key words and phrases)

I was looking into binary classification methods for classifying whether or not a given text is related to a topic that is defined by given key words and phrases (e.g. the topic is meals and key ...
0 votes
0 answers
34 views

Is it possible to use text Auto Encoders without text generation?

I have a use case where I have large texts, and a lot of it. Pretty often the sequence length exceeds 1000 tokens. I need a lower dimensional compression of the texts as an input for a classifier. The ...
  • 101
1 vote
1 answer
69 views

Handling of readability scores for short texts

I have a classification problem using emails as my dataset. I would like to use scores from various readability formulas as features for the classification. However, most of them are defined for ...
  • 518
1 vote
0 answers
59 views

How do I use a machine learning or AI algorithm on this kind of data?

I have a text data set with keyword, thematic, Sub thematic, intent Now I have to train a keyword then first classify into sub thematic, then into thematic and then into intent. the data looks like ...
  • 11
0 votes
0 answers
23 views

Text mining technique for similarity index (account for spelling mistakes)

I only use Python Context: This is my first time ever doing this so I have no idea where to start. I have a list of 20,000 words. It can contain emotes or different languages. I need to find a way to ...

1
2 3 4 5 6