Skip to main content

Questions tagged [text-classification]

For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.

Filter by
Sorted by
Tagged with
0 votes
1 answer
92 views

Text Mapping - Medicine Names

We have a problem where we have a standardized database of Medicine names. On the other hand, there is a subset of medicine names which could have spelling mistakes, different structure or hypens, ...
2 votes
1 answer
51 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
0 votes
2 answers
110 views

How to train a model to predict if 2 samples refer to the same thing?

I have 2 ddbb with around 60,000 samples each. Both have the same features (same column names) that represent particular things with text or categories (turned into numbers). Each sample in a ddbb is ...
0 votes
1 answer
19 views

SMOTE Oversampling for Text Classification with Multiple Input Features

SMOTE Oversampling for Text Classification with Multiple Input Features I have a text classification problem where the input has 2 features: a text and a language: the text is a string variable. the ...
0 votes
1 answer
87 views

Extracting structured data from semi structured data

I want to use machine learning and NLP to convert semi-structured data in text files to structured data by predicting the patterns in the files and splitting the fields for example if I have a text ...
2 votes
2 answers
785 views

How to use text classification where the training source are txt files in categorized folders?

I have 200 *.txt unique files for each folder: Each file is a lawsuit initial text separated by legal areas (folders) of public advocacy. I would like to create training data to predict new lawsuits ...
0 votes
1 answer
135 views

How to use scikit-learn to extract features from text when I only have positive and unlabeled data?

I'm looking for something similar to this https://scikit-learn.org/stable/auto_examples/text/plot_document_classification_20newsgroups.html#sphx-glr-auto-examples-text-plot-document-classification-...
0 votes
1 answer
74 views

GradientBoostingRegressor Text Classifier

I am working to build a text classifier using a Boosting method from sklearn. It is performing quite well, at around 97% accuracy on my test data. However, the problem I am seeing is that if I input ...
1 vote
1 answer
185 views

TF Keras Text Processing - Classification Model

I'm trying to put together a script that classifies comments into either adequate or inadequate. I put a question up here earlier with all my code, but I think I've isolated the problem down into the ...
1 vote
1 answer
643 views

How to match a corpus with a string of words using a TF-IDF matrix?

I am trying to match strings of words with a website that has bulletpoints whose text is most similar to it. The way I thought of doing it is to get all of the documents from each bulletpoint into one ...
2 votes
1 answer
85 views

Classification using texts as features

I want to build a classification model to match customers and products. I have a description of each product, and a description of each customer, and the label : ...
1 vote
2 answers
88 views

Confusion with using different classes in neural networks (training vs testing)

I am new to deep learning and I am confused about having a neural network trained on certain classes and tested on different ones. Suppose I want to have a convolutional neural network that learns ...
2 votes
3 answers
132 views

Classification - get some label value to check how close to another class (Python)

I am doing text classification in python with 3 alghoritms: kNN, Naive Bayes and SVM. I have 3 classes - easy, medium and hard. The accuracy is quite fine. Is there a way to check for new text its ...
3 votes
3 answers
130 views

Classification when the classification of the previous itens matter

I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem. Suppose I have a series of events that are ...
0 votes
1 answer
175 views

sklearn text analysis - dealing with missing values

I'm working on a multi-class text classification project. My goal is simple: given a "bug", I'd like to predict to which final group owner it will be assigned to. I was able to achieve ~...
0 votes
1 answer
174 views

Training a model with a series of text responses as input

I want to train a binary classifier on text -- so something like sentiment analysis, but my input vectors are going to be a series of responses from some user separated by some separator character. I ...
3 votes
0 answers
41 views

Weird behaviour when using RobERTA for text classification

I have a dataset with around 70 classes and the dataset is largely balanced ~150 samples per class. I am finetuning RoBERTA-base for 4 epochs with a ...
0 votes
0 answers
18 views

Getting nearly 100% accuracy using Binary Classification in Tensorflow but incredibly wrong prediction levels for email messages

I'm creating a Chrome Extension to read user emails via Gmail's API, and then passing in user emails to a trained Keras model in Flask to determine whether the email was written by an AI or a Human, ...
2 votes
1 answer
790 views

Should bag of words in training set include test set data when doing text classification?

I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
0 votes
1 answer
239 views

Is it possible to classify documents of corpus using labels?

I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
0 votes
1 answer
70 views

Fine-tune zero-shot classification model multi-label

I started a small project where I am trying to fine-tune a zero-shot classification model on a proprietary dataset. I was thinking to use the NLI approach, building contradiction and entailment ...
2 votes
1 answer
22 views

Text Classification with unlimited labels, Text Extraction?

I'm looking to use ML to read in a blob of text, and extract a name from that text blob. (The blob is from an OCR result from an iPhone) The text blob varies in size, but the name is always present in ...
1 vote
1 answer
148 views

Grouping profiles strings having the same words, but occurring out of order

I have a dataframe containing a column of profile types, which looks like this: ...
0 votes
0 answers
10 views

Does Fine Tuning with Custom Label Build Upon the Capability of Zero Shot Classification or Does It Train from Scratch?

The task is to classify email text bodies into exclusive categories like feedback, complaint etc. I have a labelled dataset available having about 350 samples. I have tried the ...
0 votes
0 answers
30 views

Multilabel Classification - Flat Binary Classifiers vs Hierarchical Binary Classifiers

Was researching on multi label classification to solve the problem of tagging news articles with topics and countries, where tags follow the syntax <topic>-<country>, and would like to ...
1 vote
1 answer
198 views

Datasets for String Classification

I would like to test an experimental algorithm for string classification. More precisely, the dataset should be split into a set GOOD of good strigs and a set BAD of bad strings. The algorithm should ...
0 votes
1 answer
61 views

Text classification and predictive model

I have collected a lot of data that I would like to analyse and classified. Unfortunately, they are not already labelled, so I am going to do manually. The dataset consists of texts in Italian and I ...
0 votes
1 answer
171 views

Text Classification misclassifying?

I am trying to solve a binary classification problem. My labels are abusive (1) and non-abusive (0). My dataset was imbalanced (more 1 than 0s) and I used oversampling of the minority label (i.e. 1) ...
1 vote
1 answer
196 views

Building a text classification model from scratch

I am a beginner in data science and machine learning techniques. I would need to build a model that allows me to classify texts based on sentiment analysis. Right now I only have the text and they ...
0 votes
0 answers
11 views

Is this how you would go about this NLP Project?

What do you think of these steps? And where can I find help with this project? I am in a business class and was assigned a data science problem. I was advised to seek out a coder at my school who can ...
1 vote
1 answer
156 views

What is a good approach for embedding both textual and spatial features for document classification?

I am working on a document classifier that can perform the classification based on the document structure as well. My plan is to get the word embedding as well as the word coordinates and somehow ...
0 votes
0 answers
9 views

Will hypermeters tuned on sampled dataset work for the whole dataset?

I'm doing multi-label classification on text data using BERT model. Since the dataset is huge, around 50 thousand rows, I was thinking to use stratify sampling on dataset to reduce it to around 2-4 ...
0 votes
1 answer
101 views

Aspect-Based Sentiment Analysis with Bert and Pytorch

I have a dataset of online reviews (X) with their corresponding topics (topic1 to topic5) and each topic can have 5 values (fined-grained sentiment score from 1 to 5). So, I have one X and 5 Y columns....
1 vote
1 answer
672 views

Updating a genism LDA model with new documents and topics

I have a conceptual problem that is related to a project I'm working on. I'm relatively new to the domain of NLP so this might be a poor question but I would really appreciate any help. My dataset is ...
0 votes
0 answers
25 views

Approach for Multi-class Classification of texts

I'm trying to do a project where I have paragraphs and I need to classify them into multiple labels. The dataset is around 40k rows with labels. I understand there is no one right approach but should ...
0 votes
1 answer
340 views

Text classification analysis based on similarity

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
0 votes
1 answer
127 views

How to identify the feature that make the model misclassifed in text classification

Hi I am working on social media financial THAI text classification, the problem with this one is the confused classes, the misclassified prediction has a pattern that consistent as a pair. and I want ...
2 votes
3 answers
248 views

How to identify text similarity based on training data?

I have a set of documents (1 to 11) for which the labeling is done. Lets Assume: ...
2 votes
1 answer
640 views

How to use ontologies for text classification?

I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset). My question is how ...
0 votes
1 answer
30 views

Recommendation system NLP ideas

The problem: If we have a clustering problem with lets say x groups. And each group has a document describing it, lets say 3 pages. Then we have n observations each with a smaller piece of text ...
1 vote
1 answer
301 views

Does high number of output labels affect the performance of BERT and how to handle the class imbalance issue while doing multi text classification?

I am using BERT to do multiclass text classification. The number of output classes I have to predict from is: 116 and there is high degree of class imbalance that I see. We have the following kind of ...
1 vote
1 answer
471 views

How to down\up sample text?

I have data set of 5566 samples - one column is the text of the recipe description and the other is what tax class is it. I wish to make a classifier that would classify receipts using ML only. I have ...
1 vote
1 answer
35 views

Is it methodologically correct to use the data to be used for finetuning in the pretrain phase of the BERT model?

Let us assume the training of a BERT model. An initial pre-train is performed with a large data set A. Subsequently a finetuning is performed with a dataset B which is part of A, but now with labels ...
0 votes
1 answer
121 views

Optimal input setup for character-level text classification RNN

I want to classify 500-character long text samples as to whether they look like natural language using a character-level RNN. I'm unsure as to the best way to feed the input to the RNN. Here are two ...
0 votes
1 answer
27 views

What specific problems in what domains and fields have the need to use rule-based text classification?

I wrote a rule-based keyword detection and classification program specialized in my language (Vietnamese) and would like to know where this app is useful. Here how the program work: First you input ...
1 vote
1 answer
140 views

what kind of algorithm should I use to classify the text data example given?

What kind of classification or learning algorithm that suits this kind of data example If I have to build a model using the given key words then predict column B and then to column A? what kind of ...
2 votes
2 answers
109 views

Improve text classification accuracy

Task: I am building a text classification for salary prediction for data science jobs. I want to achieve at least 70 percent accuracy. Data: Features: Consists of job descriptions of data science, ...
0 votes
0 answers
9 views

Zero-shot out-of-distribution text classification

I'm building out a pipeline that would allow me to filter out text based on whether or not the text belongs to any of the classes I've defined. I feel like one (albeit naive) approach would simply be ...
0 votes
0 answers
8 views

predict if news article belong to specific category or not?

I am still new to machine learning. I am trying to build an ML model to predict if an article belongs to a category or not. for example, I have three categories : [war, politics, and crime]. I choose ...
0 votes
1 answer
75 views

How to provide Intentional Bias towards recent examples in Text Classification?

I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...

1
2 3 4 5 6