Questions tagged [text-classification]
The text-classification tag has no usage guidance.
169
questions
0
votes
0answers
33 views
text classification - does number of features matters?
I'm working on a multi-class text classification project that aims to assign a "new bug" to his "final group assignee"
To do that I was able to extract ~17000 samples and divided ...
0
votes
0answers
8 views
keras Tokenizer usage on a whole dataframe
I've a dataframe where all its content is text based.
After separated it into features and labels, my next obvious step was to Tokenize it.
However, I can't understand how to ...
1
vote
1answer
23 views
How to match a corpus with a string of words using a TF-IDF matrix?
I am trying to match strings of words with a website that has bulletpoints whose text is most similar to it. The way I thought of doing it is to get all of the documents from each bulletpoint into one ...
0
votes
1answer
27 views
Algorithms for SMS spam detection
Which among KNN, Logistic and Naive Bayes would yield best results for SMS spam
detection? Is there any other efficient approach worth exploring.
I am planning to make a python application for SMS ...
0
votes
0answers
42 views
Improving score accuracy for multi class classification
I'm working on a multi-class text classification project.
My goal is simple: given a "bug", I'd like to predict to which final group owner it will be assigned to.
I've 6 classes overall and ...
1
vote
2answers
30 views
what make intent detection/classification different than random text classification?
I'm trying to understand what makes intents detection / classification different than random text classification. I always see examples of intents detection using a json file with the intent as a key ...
0
votes
0answers
14 views
The best way/instruments to use a communication protocol messages in hex form as input parameter for machine/deep learning (n-grams?)
I am trying to categorize server software versions based on server responses to various slightly different hex messages.
To extract the ML input parameters from these hex messages I suppose to use the ...
0
votes
0answers
11 views
Is there any benchmark dataset for unbalanced text classification?
I want to work on an unbalanced dataset for text classification (sentiment analysis, intent classifier) and hopefully, come up with an idea to improve the classification on such datasets. Is there any ...
0
votes
1answer
46 views
sklearn text analysis - dealing with missing values
I'm working on a multi-class text classification project.
My goal is simple: given a "bug", I'd like to predict to which final group owner it will be assigned to.
I was able to achieve ~...
1
vote
1answer
29 views
One-Class Text Classification
So I have a specific use case where my colleagues have kept thousands of articles across the years deemed as "Good", among hundreds of thousands of other articles deemed as bad and they didn'...
0
votes
0answers
18 views
NLP for recognising abstract concepts
I know that NLP algorithms can be trained to recognise the topic of a document or part of a document. I would like to train an algorithm to recognise certain abstract concepts. For example I want to ...
0
votes
0answers
19 views
What are good resources to learn KNN text classification using Tensorflow?
I found the KNN image classification tutorial using MNIST dataset and was able to lear how it's working.
Are there any resources that describe to apply the same KNN algorithm to text classification. I ...
1
vote
0answers
16 views
Text classification - Multiple texts for a single instance (Deep Learning)
My task is to create a classification model (e.g. DDN, RNN) which as an input should accept list of texts and should produce a binary output (good, bad). Of-course the texts will be preprocess ...
1
vote
1answer
23 views
relaying on feature during training that won't (necessarily) be available during prediction
I'm doing a little project of bugs prediction. My goal is to predict which bug will be (eventually) assigned to which relevant group (this is my label obviously).
For training, I'm relaying on a bugs ...
1
vote
0answers
17 views
What is the best feature extraction technique for text domain novelty / anomaly detection?
I am working with a text classification system. Here, my data-set has around 30 intents. But the problem is I have no system developed to handle inputs that don't go under any of the intents. So, in ...
1
vote
0answers
16 views
Most useful clustering algorithm for NER / document matrix
I have a matrix composed of documents in columns and named entities recognized in all the documents as rows.
K-means clustering has not offer me a meaningful set of clusters, and indeed one of the ...
0
votes
0answers
12 views
How to detect out-of-domain text input?
I have a text classifier which can classify around 40 classes. But the problem is there is no way to handle the case where if any user gives some input to the model which input doesn't match with any ...
1
vote
0answers
11 views
Deep learning model is giving strange metrics
I am trying to build a deep learning model for automatic short answer scoring using TensorFlow. I am using this dataset: https://www.kaggle.com/c/asap-sas/data
I am trying to build a model that suits ...
2
votes
2answers
111 views
classification of similar text input features with text output label
I hope somebody can provide guidance/input/advice on my project, where I believe AI can help.
I have a general understanding of AI, but I lack a formal training.
I've never built a neural net from ...
0
votes
0answers
40 views
Combine BertForSequenceClassificaion with Additional Features
I'm using BertForSequenceClassification + Pytorch Lightning-Flash for a text classification task.
I want to add additional features besides the text (e.g. categorical features). From what I understand,...
0
votes
0answers
22 views
classification of bugs ownership by their content (improve score for log analysis)
I'm doing a little project in which given a dataset of bugs and their relevant owner, I'd like to predict the "final owner" of a non-analyzed bug (bugs tends to assign back and forth between ...
0
votes
1answer
34 views
Text Classification misclassifying?
I am trying to solve a binary classification problem. My labels are abusive (1) and non-abusive (0).
My dataset was imbalanced (more 1 than 0s) and I used oversampling of the minority label (i.e. 1) ...
0
votes
0answers
12 views
ML model to predict values from text (a lot of training training data)
I have around 1M entries of the type:
id | big5_openness | big5_conscientiousness | big5_extraversion | big5_agreeableness | big5_neuroticism | input_text
Where <...
0
votes
1answer
17 views
Can I modify a Logistic Regression classifier to out put more than one class based on the probabilty?
I'm training a Logistic Regression classifier on text data. I found that many of my data points have more than one target class. Is it possible to modify my model to output more than one class based ...
2
votes
1answer
38 views
Should bag of words in training set include test set data when doing text classification?
I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
1
vote
1answer
23 views
How to improve my f1 score in stories analyze
I got an assignment to build a model that identify the gender of the text writer.
The assignment score will determine by my model f1_score, to get the maximum points, T need it will be at least 0.7.
I'...
0
votes
0answers
11 views
Multi-input NN Debugging
I have trained a simple multi-input NN for classification. I have 4 inputs ( one text field & other 3 categorical variables : cat1, cat2 , cat3).
For the text field, I use Glove embeddings in the ...
1
vote
2answers
34 views
Python Text Classification - Data that does not fit into any category
I am having a lot of trouble finding any kind of answers to this problem i am facing.
I have a few text classifiers that i am testing out, and they work well for data that does fit into any predefined ...
0
votes
1answer
24 views
GradientBoostingRegressor Text Classifier
I am working to build a text classifier using a Boosting method from sklearn. It is performing quite well, at around 97% accuracy on my test data. However, the problem I am seeing is that if I input ...
0
votes
2answers
34 views
Is this a tried alternative to word embedding for NLP?
I'm searching for research related to my idea, but apparently cannot articulate it well enough to the search engines to show me what's been published on this.
My idea: in a deep learning context (text ...
0
votes
1answer
50 views
Which output of a BiLSTM layer should be used for classification
I am trying to implement a BiLSTM layer for a text classification problem and using PyTorch for this.
...
1
vote
1answer
33 views
How do I split contents in a text that would include two or more different themes (context) in NLP?
For example, a text: "The airlines have affected by Corona since march 2020 a crime has been detected in Noia village this morning"
the output should be:
The airline companies have affected ...
0
votes
0answers
19 views
How to using elmo embedding for other language?
I am using the language model ELMo represent my text data as a numerical vector. This vector will be used as training data for a named entity recognition with BilSTM-CRF. My text data is not English, ...
0
votes
0answers
17 views
How to filter data samples which do not improve classifier?
I have a text dataset with noisy labels and an unbalanced shape.
There are various ways to find features which do not drive improvement in some metric, and help to prune those from the pipeline.
I ...
0
votes
1answer
75 views
How to improve LSTM accuracy on multiclass text classification?
So, I'm trying to build a LSTM model to classify multiclass text label. The goal is to make a prediction about user rating (1, 2, 3, 4, 5) based on their review.
My hyperparameter is like this:
...
1
vote
0answers
16 views
How would you optimize this Binary Text Classification model further? The data set is large (40000 texts)
I'm learning about tensorflow/keras since a couple months.
What are some methods to reduce overfitting in the later epochs or increase val. accuracy in general?
train data dim (40000,70) (reddit ...
0
votes
0answers
26 views
Can one still train a classifier with an unbalanced data set?
I want to train a binary Naive Bayes classifier. The problem is, is that I have an unbalanced set at my disposal, where the ration between the two classes is roughly 2:1 (250 examples from the first ...
1
vote
2answers
24 views
Word list as a baseline for measuring a classifier's performance?
I am working on a simple Naive Bayes classifier that categorizes text messages as either "positive" or "negative". I was told that the simplest baseline to measure the classifier's ...
0
votes
0answers
17 views
How to write decider function for multiple models
I have trained two classifiers .. Text Classification and Image Classification.
So both models gives score for each class. For example there are 3 classes. Each model give array of 3 confidence score ...
1
vote
1answer
28 views
Datasets for String Classification
I would like to test an experimental algorithm for string classification. More precisely, the dataset should be split into a set GOOD of good strigs and a set BAD of bad strings. The algorithm should ...
0
votes
1answer
24 views
How to identify the feature that make the model misclassifed in text classification
Hi I am working on social media financial THAI text classification, the problem with this one is the confused classes, the misclassified prediction has a pattern that consistent as a pair.
and I want ...
0
votes
0answers
9 views
outdoor/indoor recognition in text
For a project I should train a model that predicts whether a certain text given as input (e.g the chapter of a book) is set indoor or outdoor. I do not have any labelled data.
Could you point to ...
0
votes
0answers
15 views
What is the relation between the features of word-embeddings and the type of classifiers (linear or non-linear)?
I am doing an analysis on the results of classifiers (such as: linear SVM, RBF, Random forest, naive bayes and logistic regression) of a text classifcation problem using word-embeddings (I am using ...
0
votes
1answer
267 views
Sentence embeddings with LSTM to classify the sentences is not working
I am trying to build LSTM NN to classify the sentences. I have seen many examples where sentences are converted to word vectors using glove, word2Vec and so on here is an example of it. This solution ...
1
vote
1answer
76 views
Unsupervised Text Classification with Python: Kmeans
I am working on a project to build a text classifier of questions being asked. There are no labels provided in my data so I have chosen to go with an unsupervised approach. This solution needs to read ...
1
vote
1answer
38 views
Classification using texts as features
I want to build a classification model to match customers and products. I have a description of each product, and a description of each customer, and the label : ...
0
votes
0answers
27 views
Newspaper analysis
I'm currently working on a project, where I analyze newspaper articles about AI with NLP methods. My research question: Does newspaper coverage focus more on the advantages or disadvantages of AI?
I ...
0
votes
0answers
20 views
Use Large Existing Dataset to Extract Information From Text
What I have:
I have a large dataset of documents and their data.
So I have the text of about 1M documents, and I know, for example the invoice number, of each one.
What I need:
Is there a way to use ...
0
votes
0answers
4 views
Identifying hierarchical structures from user submitted tags
I am working with a dataset of shops, where users have submitted some number (up to 5) of "tags" for each shop, which is supposed to describe an aspect of the shop.
From the tags, I want to ...
0
votes
0answers
30 views
Training a Supervised Classifier using Continuous Data
I am trying to build a NLP classification model using methods such as XGBoost, SVM, logistic regression. The features I am trying to include are cosine similarity and LDA topic models, all of which ...
