Skip to main content

Questions tagged [text-classification]

For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.

134 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4 votes
0 answers
190 views

Bag of words: Prediction on new (out-of-sample) data

I'm working with a bag of words in R: library(tm) corpus = VCorpus(textsource) dtm = DocumentTermMatrix(corpus) dtm = as.matrix(dtm) I use the matrix ...
Peter's user avatar
  • 7,526
3 votes
0 answers
41 views

Weird behaviour when using RobERTA for text classification

I have a dataset with around 70 classes and the dataset is largely balanced ~150 samples per class. I am finetuning RoBERTA-base for 4 epochs with a ...
user1274878's user avatar
3 votes
3 answers
130 views

Classification when the classification of the previous itens matter

I have a classification problem to solve, that seems to be common but I am struggling to find the name of this task and the best way to model this problem. Suppose I have a series of events that are ...
bratao's user avatar
  • 31
3 votes
1 answer
43 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
R Sorek's user avatar
  • 53
2 votes
0 answers
134 views

how Can we add extra word embedding to the pytorch funnel transformer?

i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow. first i created laserembedding like this : ...
Syed Mobassir's user avatar
2 votes
0 answers
76 views

sentence type classification

I want to classify the sentences in my dataset as declarative, interrogative, imperative and ...
Mohsen Mahmoodzadeh's user avatar
2 votes
0 answers
74 views

Multi-Class Document Classification with both known and un-known classes

Currently, I am building a multi-class document classifier which has to classify either 3 known classes, namely "Financial Report", "Insurance_Sheet", "Endorsement", and ...
Quan Nguyen Ha's user avatar
2 votes
0 answers
62 views

Best practices to train a transformer text classifier to predict/handle unseen labels

I fine-tuned a RoBERTa sequence classifier to classify paragraphs of certain documents using labeled paragraphs only (and skipping paragraphs with no label given). The model was validated and tested ...
Arthuro's user avatar
  • 101
2 votes
0 answers
54 views

Classification Texte with naive bayes complement

Currently I am on a text classification project, the goal is to classify a set of CVs according to 13 classes. I use the bayes algorithm (ComplementNB), in my tests it is the model that gives the ...
amroun lysa's user avatar
2 votes
0 answers
350 views

Creating a Sentiment dictionary from scratch

I am analyzing Arabic textual data from a social media forum discussing economic issues such as labor unions. I am using a package that classifies as negative, positive, or neutral. For instance, the ...
maldini425's user avatar
2 votes
1 answer
51 views

Classifying short strings of text with additional context

I have a list of short strings each identifying a city. Misspellings are very common. The example below shows some of these short strings, along with the correct city they're supposed to match. ...
Jivan's user avatar
  • 165
2 votes
1 answer
85 views

Classification using texts as features

I want to build a classification model to match customers and products. I have a description of each product, and a description of each customer, and the label : ...
sgduran91's user avatar
2 votes
0 answers
28 views

Does Keras allow using independant classifier

I have spam data set classified into 0:ham or 1:spam. I created the Embedding layer with Keras and then I used Conv1D following with other layers. My question is about adding a classifier like Random ...
Mai Ahmed's user avatar
2 votes
3 answers
248 views

How to identify text similarity based on training data?

I have a set of documents (1 to 11) for which the labeling is done. Lets Assume: ...
Shivam Agrawal's user avatar
2 votes
1 answer
640 views

How to use ontologies for text classification?

I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset). My question is how ...
RFAI's user avatar
  • 121
2 votes
2 answers
785 views

How to use text classification where the training source are txt files in categorized folders?

I have 200 *.txt unique files for each folder: Each file is a lawsuit initial text separated by legal areas (folders) of public advocacy. I would like to create training data to predict new lawsuits ...
celsowm's user avatar
  • 121
1 vote
0 answers
30 views

How to implement Ant Lion Optimization (ALO) feature selection for KNN Classification Problems?

I have been assigned for a project related to text data classification, i have preprocess and vectorized the data with TF-IDF. For feature selection i am using pyMetahueristic library to implement ALO ...
Physics69's user avatar
1 vote
0 answers
12 views

Boosting the effect of some of the features in SVM

I'm doing text classification with SVM. I'm using Tfidf vectorization. In addition to the text vectors, I have a context data denoting the possible outcomes of the prediction. For example, I have a ...
cuneyttyler's user avatar
1 vote
0 answers
35 views

NLP - F1 score for positive class drops to 0 after data augmentation

I'm working on a 3-class text classification problem where my initial class distribution looked like this: positive: 50% negative: 25% and neutral: 25% And training on a model on this slightly ...
AnonymousMe's user avatar
1 vote
1 answer
45 views

Handling missing value in column having textual data

I have been working on a supervised ML use case where dataset has Numerical (Price), Categorical(Category) and Textual data(Description) as features. Description feature has about 30% missing values. ...
shobhit kumar's user avatar
1 vote
0 answers
21 views

How to design and use CNNs for sentence classification?

I'm playing around with using CNNs for sentence classification. Basically all models I found implement the same model proposed in Convolutional Neural Networks for Sentence Classification (Kim, 2014), ...
Christian's user avatar
  • 181
1 vote
0 answers
123 views

Text + tabular data for classification

I have a dataset of "mixed" types e.g text | date | amount | supplier -----+--------+---------+------------ where ...
CutePoison's user avatar
1 vote
0 answers
61 views

How do I use a machine learning or AI algorithm on this kind of data?

I have a text data set with keyword, thematic, Sub thematic, intent Now I have to train a keyword then first classify into sub thematic, then into thematic and then into intent. the data looks like ...
j123's user avatar
  • 11
1 vote
0 answers
184 views

How to generate labels for the text data given in excel files?

I have a data given like this How do I convert this kind of data into a organized dataset like news group dataset https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html
User123456's user avatar
1 vote
0 answers
15 views

Is It Fundamentally Correct To The Text Classification Model To Train First Without Pre-Trained Word Vectors And Then With Pre-Trained Word Vectors?

Is this solution fundamentally correct to the text classification (sentiment analysis) model to train it by these three steps: train the model without pre-trained word vectors untill reaches the ...
Soroush Mirzaei's user avatar
1 vote
0 answers
16 views

Is This Solution Fundamentally Correct To The Text Classification Model With Pre-Trained Word Vectors?

Is it fundamentally correct to training text classification (sentiment analysis) model with pre-trained word vectors; first with the locked embedding layer, and then train again with locked additional ...
Soroush Mirzaei's user avatar
1 vote
0 answers
51 views

Contextual word embeddings from pretrained word2vec vectors

I would like to create word embeddings that take context into account, so the vector of the word Jaguar [animal] would be different from the word Jaguar [car brand]. As you know, word2vec only gives ...
amks1212's user avatar
1 vote
1 answer
470 views

How to down\up sample text?

I have data set of 5566 samples - one column is the text of the recipe description and the other is what tax class is it. I wish to make a classifier that would classify receipts using ML only. I have ...
JamseGoldman's user avatar
1 vote
0 answers
105 views

Getting the keywords of text classification prediction in real time

I am using a BERT based text classification model to classify sentences/documents. My use case is like I need to highlight the words that are responsible for classification of a particular class. This ...
sampath kumaran's user avatar
1 vote
0 answers
110 views

Tensorflow text classification in R using 3 classes - Error in py_call_impl(callable, dots\$args, dots\$keywords)

I'm working on a text classification problem that classifies some tweets into one of three labels. I have two columns in my dataset: Score column with the value of 0 (negative), 1 (positive) or 2 (...
vvalentina96's user avatar
1 vote
0 answers
19 views

Is there any method to separate text into paragraphs of similar context

I want to split my text that has various sentences of different context mixed into separate paragraphs of similar context. for example, if my text is: ...
Tushar Patil's user avatar
1 vote
0 answers
25 views

What are the best ways to convert different parts of speech to noun?

I am working on a topic modelling task. I want to convert different parts of speech such as adjectives, verbs to noun. What are the best ways of doing this? I have tried lemmatization using NLTK ...
user16584277's user avatar
1 vote
0 answers
19 views

Any transformer model (NLP) for code classification?

Does any Transformer (NLP) that is suitable for code classification tasks exist? For example, I have a lot of source codes of various categories (driver, game, email client, etc.). I want to ...
askar's user avatar
  • 11
1 vote
1 answer
672 views

Updating a genism LDA model with new documents and topics

I have a conceptual problem that is related to a project I'm working on. I'm relatively new to the domain of NLP so this might be a poor question but I would really appreciate any help. My dataset is ...
Tanay Roman's user avatar
1 vote
0 answers
326 views

NER prections with distilbert transformer model

I am trying to extract 'agreement date' label from a corpus of legal contracts. In the train dataset, I used pytorch-transformer model to train. ...
Jay's user avatar
  • 56
1 vote
0 answers
367 views

Is it possible to fine-tune a (Spanish RoBERTa) model for a different task?

I'm doing sentiment analysis of Spanish tweets. After reviewing some of the recent literature, I've seen that there's been a most recent effort to train a RoBERTa model exclusively on Spanish text. It ...
LeLuc's user avatar
  • 131
1 vote
1 answer
39 views

Which ML algorithm is best works on text data and the reason behind it? Also, which metrics is used for testing performance of model?

I am working on a project - 'sentiment analysis of tweets.' There are 5 different sentiments - extremely negative, negative, neutral, positive, and extremely positive. So it is basically the NLP ...
Sumeet Agrawal's user avatar
1 vote
1 answer
643 views

How to match a corpus with a string of words using a TF-IDF matrix?

I am trying to match strings of words with a website that has bulletpoints whose text is most similar to it. The way I thought of doing it is to get all of the documents from each bulletpoint into one ...
sangstar's user avatar
  • 133
1 vote
0 answers
32 views

Text classification - Multiple texts for a single instance (Deep Learning)

My task is to create a classification model (e.g. DDN, RNN) which as an input should accept list of texts and should produce a binary output (good, bad). Of-course the texts will be preprocess ...
Kuba's user avatar
  • 11
1 vote
1 answer
34 views

What is the best feature extraction technique for text domain novelty / anomaly detection?

I am working with a text classification system. Here, my data-set has around 30 intents. But the problem is I have no system developed to handle inputs that don't go under any of the intents. So, in ...
hafiz031's user avatar
  • 121
1 vote
0 answers
210 views

Most useful clustering algorithm for NER / document matrix

I have a matrix composed of documents in columns and named entities recognized in all the documents as rows. K-means clustering has not offer me a meaningful set of clusters, and indeed one of the ...
Juan Luis Chulilla's user avatar
1 vote
0 answers
33 views

Deep learning model is giving strange metrics

I am trying to build a deep learning model for automatic short answer scoring using TensorFlow. I am using this dataset: https://www.kaggle.com/c/asap-sas/data I am trying to build a model that suits ...
youssef manyalawy's user avatar
1 vote
1 answer
47 views

How to improve my f1 score in stories analyze

I got an assignment to build a model that identify the gender of the text writer. The assignment score will determine by my model f1_score, to get the maximum points, T need it will be at least 0.7. I'...
Bar Benezri's user avatar
1 vote
0 answers
21 views

How would you optimize this Binary Text Classification model further? The data set is large (40000 texts)

I'm learning about tensorflow/keras since a couple months. What are some methods to reduce overfitting in the later epochs or increase val. accuracy in general? train data dim (40000,70) (reddit ...
dat boi's user avatar
  • 11
1 vote
1 answer
198 views

Datasets for String Classification

I would like to test an experimental algorithm for string classification. More precisely, the dataset should be split into a set GOOD of good strigs and a set BAD of bad strings. The algorithm should ...
verifying's user avatar
  • 111
1 vote
0 answers
27 views

Extracting layer output from Classification model of SimpleTransformer

I have fine tuned a bert base model for text classification task. Now, I want to extract hidden layer output so as to combine this output with other features to train a random forest model. Problem ...
SK Singh's user avatar
1 vote
1 answer
156 views

What is a good approach for embedding both textual and spatial features for document classification?

I am working on a document classifier that can perform the classification based on the document structure as well. My plan is to get the word embedding as well as the word coordinates and somehow ...
Akas Antony's user avatar
1 vote
0 answers
16 views

auto updating text comparison model

I have a need to create a model that compares and groups distinct snippets of text based on keywords. I can extract similar keywords with NLP methods and simply comparing sentence text. I want these ...
eerick's user avatar
  • 121
1 vote
0 answers
75 views

Sklearn - multiclass text classification

I took a challenge to classify bugs (by scanning their logs) into a different groups (classes). I've already accomplished part of the task by fetching + cleaning the data, but I'm now stuck on getting ...
Ben's user avatar
  • 209
1 vote
0 answers
665 views

SVM on BERT-Embeddings with very small dataset does not converge

I am trying to reproduce the results from this paper where they use a linear SVM on top of BERT-Embeddings for text-classification. They use the average of the token-embeddings which results in a 768 ...
chefhose's user avatar