Questions tagged [text-classification]

For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.

Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

Is it possible to use text Auto Encoders without text generation?

I have a use case where I have large texts, and a lot of it. Pretty often the sequence length exceeds 1000 tokens. I need a lower dimensional compression of the texts as an input for a classifier. The ...
  • 101
1 vote
1 answer
21 views

Handling of readability scores for short texts

I have a classification problem using emails as my dataset. I would like to use scores from various readability formulas as features for the classification. However, most of them are defined for ...
  • 121
1 vote
0 answers
52 views

How do I use a machine learning or AI algorithm on this kind of data?

I have a text data set with keyword, thematic, Sub thematic, intent Now I have to train a keyword then first classify into sub thematic, then into thematic and then into intent. the data looks like ...
  • 11
0 votes
0 answers
12 views

Text mining technique for similarity index (account for spelling mistakes)

I only use Python Context: This is my first time ever doing this so I have no idea where to start. I have a list of 20,000 words. It can contain emotes or different languages. I need to find a way to ...
0 votes
1 answer
24 views

Are you supposed to clean new data before it is fed to a machine learning model?

I have train/test data for my text classification problem. I have used them to create and test several ML models (LogisticRegression, RandomForest, and LinearSVC). The train and test data consist of ...
  • 143
1 vote
2 answers
46 views

Extend BERT or any transformer model using manual features

I have been doing a thesis in my citation classifications. I just implemented Bert model for the classification of citations. I have 4 output classes and I give an input sentence and my model returns ...
0 votes
1 answer
29 views

Model for detecting contact information in text

Is there a SOTA solution for finding texts with contact information (phone numbers, social media links, etc.)? I know that this task is advised to solve by regular expressions, I've already tried it ...
0 votes
0 answers
15 views

How do I use TF-IDF for set of keywords?

I have a set of keyword K = {K1,K2,K3,...} K1 = (president governor) k2 = (foot ball players goal) K3=(Hero Heroine song singer) etc. like these and each K1...Kn belongs to some category like in ...
  • 11
1 vote
0 answers
31 views

How to generate labels for the text data given in excel files?

I have a data given like this How do I convert this kind of data into a organized dataset like news group dataset https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html
0 votes
0 answers
17 views

Which model to use to classify a text to a topic, among a list of topics?

I want to find to which topics a text belongs. A topic is a question (~ 15 words) nb_topics = 250 nb_texts ~ 15000 The texts and topics are in French How can I do this with BERT or GPT-3? Any ...
  • 101
1 vote
1 answer
37 views

what kind of algorithm should I use to classify the text data example given?

What kind of classification or learning algorithm that suits this kind of data example If I have to build a model using the given key words then predict column B and then to column A? what kind of ...
0 votes
0 answers
22 views

Why would you use document embeddings for text classification?

I am looking into text classification and I am thinking of using the standard text classifier in the flair library. They have the following code in this tutorial: ...
2 votes
0 answers
16 views

how Can we add extra word embedding to the pytorch funnel transformer?

i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow. first i created laserembedding like this : ...
0 votes
1 answer
10 views

Model for binary classification of links as "Article" or "Other"

I'm creating a web crawler which must: Fetch a web page. Parse all <a> tags with hrefs on the page. Classify the tags as either article (Meaning the link ...
0 votes
0 answers
24 views

Classify book titles into categories

I have 10,000 book names sorted into 50 categories. This is my training data. I want to sort 2,000 or more names to sort with more incoming. What is the simplest way I could do this? So my training ...
0 votes
1 answer
51 views

Topic modelling or Keyword extraction for a small dataset

I am working on a project where I have a dataset which contains very less data. These are the comments of people. I have only 130 lines with 10 words per line. My goal is to identify the common topics ...
0 votes
1 answer
48 views

Limitations of NLP BERT model for sentiment analysis

I am reading a paper, where the authors assess online public sentiment in China in response tot the government's policies during Covid-19, using a Chinese BERT model. The author's objective is not ...
0 votes
1 answer
25 views

LinearSVC training time with CountVectorizer and HashingVectorizer

I am currently trying to build a text classifier and I am experimenting with different settings. Specifically, I am extracting my features with a CountVectorizer ...
0 votes
0 answers
42 views

NLP Text Classification Model with defined context / intent

This is more of a guideline question rather than a technical query. I am looking to create a classification model that classifies documents based on a specific list of strings. However, as it turns ...
  • 101
0 votes
0 answers
7 views

Are there known meanings or topics associated with pretrained word embeddings?

I've been using some pre-trained word embeddings (Glove, FastText, word2vec etc.) in text classification, with the averaged 300 dimensions of tags and sentences as input features. Normally it's useful ...
0 votes
0 answers
6 views

Combining a BoW vector with other features: effectivity and feature value ranges

I am working with a Support Vector Machine to predict class prevalence in a binary classification problem. The model will take a sparse representation of an instance as input, where the number of the ...
1 vote
0 answers
8 views

Is It Fundamentally Correct To The Text Classification Model To Train First Without Pre-Trained Word Vectors And Then With Pre-Trained Word Vectors?

Is this solution fundamentally correct to the text classification (sentiment analysis) model to train it by these three steps: train the model without pre-trained word vectors untill reaches the ...
1 vote
0 answers
13 views

Is This Solution Fundamentally Correct To The Text Classification Model With Pre-Trained Word Vectors?

Is it fundamentally correct to training text classification (sentiment analysis) model with pre-trained word vectors; first with the locked embedding layer, and then train again with locked additional ...
3 votes
1 answer
217 views

Performing a text classification based on a dictionary

I have been given a kind of dictionary which maps a category with a set of certain strings. A sample of the dictionary is given below: This is all I have, there is no other data. There are around 46 ...
1 vote
1 answer
35 views

How to deal with spelling errors in NLP classfier (low resource language)

I know there are questions on how to deal with spelling error NLP - but the question and solution are mainly focused on English where there are tons of library for spell-correction. Here I am curious ...
0 votes
0 answers
29 views

Is TF-IDF for text classification transferable between corpuses?

I am using TF-IDF for text classification and my solution works well according to the performance metric of my choice (F1 macro). To speed up the training process I have used PCA to reduce the ...
0 votes
0 answers
28 views

Batch size to avoid overfitting

I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't ...
0 votes
0 answers
6 views

Text2Slide multiclass classification

I am considering an idea of stitching together a slide deck based on text input, e.g. given: ...
0 votes
0 answers
19 views

Using a fine-tuned model for a different dataset

I have a dataset of different sentences from news articles which I need to classify by their sentiment. For that goal I'm planning to use a fine-tuned model which was fine-tuned on different datasets, ...
  • 1
2 votes
1 answer
81 views

Is there anyway to classify the category on give amazon reviews using python

I am trying to find a model or way to classify text which falls into a category and its a positive or negative feedback. For ex. we have three columns Review : Camera's not good battery backup is not ...
1 vote
0 answers
15 views

Contextual word embeddings from pretrained word2vec vectors

I would like to create word embeddings that take context into account, so the vector of the word Jaguar [animal] would be different from the word Jaguar [car brand]. As you know, word2vec only gives ...
0 votes
0 answers
83 views

Can i use Transformer-XL for text classification task?

I want to use transformer xl for text classification tasks. But I don't know the architect model for the text classification task. I use dense layers with activation softmax for logits output from the ...
  • 1
0 votes
0 answers
157 views

Language Detection using pycld2

I am trying to use the pycld2 package to detect multiple languages in text. This package provides Python bindings for the Compact Language Detect 2 (CLD2) This is the example I am testing out: ...
  • 11
0 votes
2 answers
59 views

Suggestions for a multi-class text classification model with a large number of classes?

I was working on a text classification problem where I currently have around 40-45 different labels. The input is a text sentence with a keyword. For e.g. ...
0 votes
1 answer
47 views

Ideal Windows Size in Pk Evaluation Metric

I am very new to nlp. I am doing a text segmentation task and for evaluating my model I need to calculate Pk and Windiff scores. My question is what is the ideal value for window size (k) for Pk score ...
  • 1
0 votes
0 answers
9 views

Naive Bayes TfidfVectorizer predicts everything to one class

I'm trying to run Multinomial Bayes classificator on various balanced data sets and comparing 2 different vectorizers: TfidfVectorizer and CountVectorizer. I have 3 classes: NEG, NEU and POS. I have ...
2 votes
0 answers
36 views

sentence type classification

I want to classify the sentences in my dataset as declarative, interrogative, imperative and ...
0 votes
0 answers
6 views

predictive effect in the classification made according to the comments in different fields

I want to do a classification through comments categorized in 4 areas(X,Y,Z,M). Categorizing the product as good or bad based on the comments in the fields X, Y, Z, M. How can I follow a path to see ...
  • 9
0 votes
1 answer
21 views

How can I implement text classification for this problem?

Given a collection of documents - each corresponding to some economic entity - I am looking to extract information and populate a table with predetermined headings. I have a small sample of this ...
  • 11
0 votes
1 answer
26 views

How to provide Intentional Bias towards recent examples in Text Classification?

I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
2 votes
0 answers
24 views

Multi-Class Document Classification with both known and un-known classes

Currently, I am building a multi-class document classifier which has to classify either 3 known classes, namely "Financial Report", "Insurance_Sheet", "Endorsement", and ...
0 votes
0 answers
39 views

Commercial product name classification with product id

I want to give the probability of how the entered name (user 1) for product X matches with the names (historical names from all users) of product X. I have data with the following structure: while ...
0 votes
0 answers
17 views

Deep Regression Ensembles(DRE) - text analysis

I read an article about Deep Regression Ensembles(DRE), which can outperform DNN using SDG. My question is could I use DRE in text classification? (for example, I can use it instead of LDA) What about ...
1 vote
1 answer
40 views

How to down\up sample text?

I have data set of 5566 samples - one column is the text of the recipe description and the other is what tax class is it. I wish to make a classifier that would classify receipts using ML only. I have ...
0 votes
0 answers
7 views

Removing footers from text scraped from news sites

I am wondering if anyone knows of libraries out there that will remove footer material from text scraped from news sites, like the material in italics below: “Go, I’ll catch up with you,” the woman ...
  • 101
0 votes
0 answers
9 views

Text segment identification without clear boundaries

I am considering the problem of creating a supervised machine learning algorithm able to identify segments of text that are of interest, but where the segments will not, in general, coincide with a ...
  • 247
0 votes
1 answer
32 views

Training a model with a series of text responses as input

I want to train a binary classifier on text -- so something like sentiment analysis, but my input vectors are going to be a series of responses from some user separated by some separator character. I ...
  • 133
1 vote
1 answer
365 views

What are the exact differences between Word Embedding and Word Vectorization?

I am learning NLP. I have tried to figure out the exact difference between Word Embedding and Word Vectorization. However, seems like some articles use these words interchangeably. But I think there ...
  • 13
2 votes
1 answer
52 views

A multi label text classification problem

I'm looking to solve a multi label text classification problem but I don't really know how to formulate it correctly so I can look it up.. Here is my problem : Say I have the document ...
  • 35
1 vote
1 answer
23 views

Distinguishing text with opposite meanings in SVM (False Information Detection)

I am currently working on a Binary Text Classification Model (False Information Detection) using Support Vector Machine and used TF-IDF as text vectorizer in Python. I have already tried training the ...

1
2 3 4 5 6