Questions tagged [text-classification]
For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.
195
questions
1
vote
0answers
15 views
Nested Classification in Tensorflow
I am working on a text classification problem that has an output structure that one could consider nested in the sense that similar child labels fall under the umbrella of parent labels, which are ...
1
vote
1answer
22 views
HuggingFace hate detection model
I am trying to train and evaluate a hate detection model using the HuggingFace Transformers library and this dataset. Model performance is secondary, just trying to get it going. I have preprocessed ...
0
votes
1answer
17 views
Probability of success in rule-based text classifier [closed]
I've created a rule-based text classifier similar to the example shown at this link.
My question is the following:
Is there a study on the sample size that ensures a high percentage of success in the ...
1
vote
2answers
35 views
What effect repetitive data will have on the performance of the model
I understand that my question is very broad and that the correct answer may depend on various things. I want to get an idea in general what we may expect if we have repetitive data in our dataset. ...
0
votes
0answers
14 views
Sentiment Analysis on Software Engineering texts [closed]
What are the possible ways to improve sentiment dictionaries to analyse SE texts? There are several SE specific sentiment dictionaries but cannot expect much accuracy when analysing open-source ...
1
vote
0answers
18 views
NER prections with distilbert transformer model
I am trying to extract 'agreement date' label from a corpus of legal contracts. In the train dataset, I used pytorch-transformer model to train.
...
1
vote
1answer
42 views
How to improve the result of f1 on imbalanced dataset
I have a dataset in which these are the distribution of the data:
Neutral. 15000
Negative 3000
positive 2000
And I am mostly interested to improve the ...
2
votes
0answers
20 views
Classifying short strings of text with additional context
I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match.
...
0
votes
0answers
12 views
BERT Text Classification Model gives error
I am fine-tuning a BERT Model for text classification with Tensorflow. Here is my code for building the model:
...
0
votes
0answers
11 views
1
vote
0answers
14 views
Merging models: using a named entity recognition model to annotate data on a different dataset [closed]
Lets say we have two trained models Ma and Mb which were trained with different datasets in a Named Entity Recognition task. Those datasets A and B contain different document and also variables or ...
1
vote
0answers
25 views
Is it possible to fine-tune a (Spanish RoBERTa) model for a different task?
I'm doing sentiment analysis of Spanish tweets.
After reviewing some of the recent literature, I've seen that there's been a most recent effort to train a RoBERTa model exclusively on Spanish text. It ...
1
vote
1answer
17 views
Which ML algorithm is best works on text data and the reason behind it? Also, which metrics is used for testing performance of model?
I am working on a project - 'sentiment analysis of tweets.' There are 5 different sentiments - extremely negative, negative, neutral, positive, and extremely positive. So it is basically the NLP ...
0
votes
2answers
35 views
0
votes
2answers
42 views
2
votes
1answer
82 views
1
vote
1answer
21 views
Binary document classification using keywords for a very small dataset
I have a set of 150 documents with their assigned binary class. I also have 1000 unlabeled documents. Each document is about the length of a journal paper. Each class has 15 associated keywords.
I ...
0
votes
1answer
22 views
how to filter out and discard irrelevant tweets in simplest way possible
I have lot of tweets and from which i need to filter out and discard irrelevant tweets. the criteria for a tweet to be irrelevant is very simple. if all that a tweet has is emojis or a single hastag ...
0
votes
0answers
29 views
Question about text classification without labeled data
I am working on a text classifier but at the moment I'm quite lost on what to do.
The classes form a tree with three levels, for example, class A (level 1), class A.1 (level 2, subclass of A), and ...
2
votes
2answers
88 views
Best approach for text classification of phrases with little syntactic difference
So I have the task of classifying sentences based on their level of 'change talk' shown. Change talk is a psychology term used in counseling sessions to express how much the client wants to change ...
0
votes
0answers
17 views
Comments Moderation/Profanity Filtering
Just a brief background. I am working on a project for a live-streaming app and we want to improve our live comments moderation using machine learning. The problem is that it over filter/block words ...
0
votes
2answers
31 views
How to stay up to date in NLP and use the best approaches?
There are many fast advancements in NLP field, BERT, RoBERTa, ALBERT, and XLNe, and no one can check the news or papers daily. Is there any way or site that keeps track of all these new developments ...
2
votes
1answer
25 views
For text classification, would a BoW or Word Embeddings based model ever be better than a Language Model?
I've done a bit of research, with this being the best as far as objectively measuring quality, but wanted to ask from a theoretical perspective if BoW-based models (e.g. using TF-IDF) or word ...
0
votes
0answers
6 views
Choosing an explainable embedding and classifier when each document only have one sentence
I have dataset with corpus of 20K documents.
Each document is a short 1 sentences.
I need to classify each sentence in 0/1 classes as well as being able to point exactly what words are responsible for ...
0
votes
0answers
12 views
Is it possible to train a model for sentiment analysis with data that has been labeled with VADER?
I want to perform sentiment analysis on a selection of tweets regarding vaccination.
The tweets I find are either unlabeled or have been labeled using VADER or TextBlob.
I am wondering if it makes ...
1
vote
1answer
28 views
In Text Classification if I get similar performance with 100 features and 200 features, which model should I go ahead with?
I have built two text classifier models, one has 200 features the other has 100 features (reduced to 100 from 200 after feature selection). I see similar performances in both. Which model should I go ...
0
votes
0answers
26 views
Training and Evaluating BERT and XLNET
I am thinking about a project and have a few questions before I accept it. Would be grateful I anyone experienced of you could give me some advice.
In the project, I have been given a data set with (...
0
votes
0answers
35 views
text classification - does number of features matters?
I'm working on a multi-class text classification project that aims to assign a "new bug" to his "final group assignee"
To do that I was able to extract ~17000 samples and divided ...
0
votes
0answers
39 views
keras Tokenizer usage on a whole dataframe
I've a dataframe where all its content is text based.
After separated it into features and labels, my next obvious step was to Tokenize it.
However, I can't understand how to ...
1
vote
1answer
69 views
How to match a corpus with a string of words using a TF-IDF matrix?
I am trying to match strings of words with a website that has bulletpoints whose text is most similar to it. The way I thought of doing it is to get all of the documents from each bulletpoint into one ...
0
votes
1answer
36 views
Algorithms for SMS spam detection
Which among KNN, Logistic and Naive Bayes would yield best results for SMS spam
detection? Is there any other efficient approach worth exploring.
I am planning to make a python application for SMS ...
0
votes
0answers
44 views
Improving score accuracy for multi class classification
I'm working on a multi-class text classification project.
My goal is simple: given a "bug", I'd like to predict to which final group owner it will be assigned to.
I've 6 classes overall and ...
1
vote
2answers
44 views
what make intent detection/classification different than random text classification?
I'm trying to understand what makes intents detection / classification different than random text classification. I always see examples of intents detection using a json file with the intent as a key ...
0
votes
0answers
17 views
The best way/instruments to use a communication protocol messages in hex form as input parameter for machine/deep learning (n-grams?)
I am trying to categorize server software versions based on server responses to various slightly different hex messages.
To extract the ML input parameters from these hex messages I suppose to use the ...
0
votes
0answers
15 views
Is there any benchmark dataset for imbalanced text classification?
I would like to work on an unbalanced dataset for text classification (sentiment analysis, intent classifier) and hopefully, come up with an idea to improve the classification on such datasets. Is ...
0
votes
1answer
47 views
sklearn text analysis - dealing with missing values
I'm working on a multi-class text classification project.
My goal is simple: given a "bug", I'd like to predict to which final group owner it will be assigned to.
I was able to achieve ~...
1
vote
1answer
41 views
One-Class Text Classification
So I have a specific use case where my colleagues have kept thousands of articles across the years deemed as "Good", among hundreds of thousands of other articles deemed as bad and they didn'...
0
votes
0answers
23 views
NLP for recognising abstract concepts
I know that NLP algorithms can be trained to recognise the topic of a document or part of a document. I would like to train an algorithm to recognise certain abstract concepts. For example I want to ...
0
votes
0answers
23 views
What are good resources to learn KNN text classification using Tensorflow?
I found the KNN image classification tutorial using MNIST dataset and was able to lear how it's working.
Are there any resources that describe to apply the same KNN algorithm to text classification. I ...
1
vote
0answers
17 views
Text classification - Multiple texts for a single instance (Deep Learning)
My task is to create a classification model (e.g. DDN, RNN) which as an input should accept list of texts and should produce a binary output (good, bad). Of-course the texts will be preprocess ...
1
vote
1answer
24 views
relaying on feature during training that won't (necessarily) be available during prediction
I'm doing a little project of bugs prediction. My goal is to predict which bug will be (eventually) assigned to which relevant group (this is my label obviously).
For training, I'm relaying on a bugs ...
1
vote
0answers
18 views
What is the best feature extraction technique for text domain novelty / anomaly detection?
I am working with a text classification system. Here, my data-set has around 30 intents. But the problem is I have no system developed to handle inputs that don't go under any of the intents. So, in ...
1
vote
0answers
38 views
Most useful clustering algorithm for NER / document matrix
I have a matrix composed of documents in columns and named entities recognized in all the documents as rows.
K-means clustering has not offer me a meaningful set of clusters, and indeed one of the ...
0
votes
0answers
17 views
How to detect out-of-domain text input?
I have a text classifier which can classify around 40 classes. But the problem is there is no way to handle the case where if any user gives some input to the model which input doesn't match with any ...
1
vote
0answers
12 views
Deep learning model is giving strange metrics
I am trying to build a deep learning model for automatic short answer scoring using TensorFlow. I am using this dataset: https://www.kaggle.com/c/asap-sas/data
I am trying to build a model that suits ...
2
votes
2answers
117 views
classification of similar text input features with text output label
I hope somebody can provide guidance/input/advice on my project, where I believe AI can help.
I have a general understanding of AI, but I lack a formal training.
I've never built a neural net from ...
0
votes
0answers
47 views
Combine BertForSequenceClassificaion with Additional Features
I'm using BertForSequenceClassification + Pytorch Lightning-Flash for a text classification task.
I want to add additional features besides the text (e.g. categorical features). From what I understand,...
0
votes
0answers
24 views
classification of bugs ownership by their content (improve score for log analysis)
I'm doing a little project in which given a dataset of bugs and their relevant owner, I'd like to predict the "final owner" of a non-analyzed bug (bugs tends to assign back and forth between ...
0
votes
1answer
36 views
Text Classification misclassifying?
I am trying to solve a binary classification problem. My labels are abusive (1) and non-abusive (0).
My dataset was imbalanced (more 1 than 0s) and I used oversampling of the minority label (i.e. 1) ...
0
votes
0answers
12 views
ML model to predict values from text (a lot of training training data)
I have around 1M entries of the type:
id | big5_openness | big5_conscientiousness | big5_extraversion | big5_agreeableness | big5_neuroticism | input_text
Where <...
