Questions tagged [text-classification]
For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.
255
questions
0
votes
0
answers
22
views
Is it possible to use text Auto Encoders without text generation?
I have a use case where I have large texts, and a lot of it. Pretty often the sequence length exceeds 1000 tokens. I need a lower dimensional compression of the texts as an input for a classifier. The ...
- 101
1
vote
1
answer
21
views
Handling of readability scores for short texts
I have a classification problem using emails as my dataset.
I would like to use scores from various readability formulas as features for the classification. However, most of them are defined for ...
- 121
1
vote
0
answers
52
views
How do I use a machine learning or AI algorithm on this kind of data?
I have a text data set with keyword, thematic, Sub thematic, intent Now I have to train a keyword then first classify into sub thematic, then into thematic and then into intent. the data looks like ...
- 11
0
votes
0
answers
12
views
Text mining technique for similarity index (account for spelling mistakes)
I only use Python
Context:
This is my first time ever doing this so I have no idea where to start.
I have a list of 20,000 words. It can contain emotes or different languages. I need to find a way to ...
- 113
0
votes
1
answer
24
views
Are you supposed to clean new data before it is fed to a machine learning model?
I have train/test data for my text classification problem. I have used them to create and test several ML models (LogisticRegression, RandomForest, and LinearSVC).
The train and test data consist of ...
- 143
1
vote
2
answers
46
views
Extend BERT or any transformer model using manual features
I have been doing a thesis in my citation classifications. I just implemented Bert model for the classification of citations. I have 4 output classes and I give an input sentence and my model returns ...
- 11
0
votes
1
answer
29
views
Model for detecting contact information in text
Is there a SOTA solution for finding texts with contact information (phone numbers, social media links, etc.)?
I know that this task is advised to solve by regular expressions, I've already tried it ...
0
votes
0
answers
15
views
How do I use TF-IDF for set of keywords?
I have a set of keyword
K = {K1,K2,K3,...}
K1 = (president governor) k2 = (foot ball players goal) K3=(Hero Heroine song singer)
etc. like these
and each K1...Kn belongs to some category like in ...
- 11
1
vote
0
answers
31
views
How to generate labels for the text data given in excel files?
I have a data given like this
How do I convert this kind of data into a organized dataset like news group dataset https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html
- 21
0
votes
0
answers
17
views
Which model to use to classify a text to a topic, among a list of topics?
I want to find to which topics a text belongs.
A topic is a question (~ 15 words)
nb_topics = 250
nb_texts ~ 15000
The texts and topics are in French
How can I do this with BERT or GPT-3? Any ...
- 101
1
vote
1
answer
37
views
what kind of algorithm should I use to classify the text data example given?
What kind of classification or learning algorithm that suits this kind of data example
If I have to build a model using the given key words then predict column B and then to column A?
what kind of ...
- 21
0
votes
0
answers
22
views
Why would you use document embeddings for text classification?
I am looking into text classification and I am thinking of using the standard text classifier in the flair library. They have the following code in this tutorial:
...
- 143
2
votes
0
answers
16
views
how Can we add extra word embedding to the pytorch funnel transformer?
i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow.
first i created laserembedding like this :
...
- 121
0
votes
1
answer
10
views
Model for binary classification of links as "Article" or "Other"
I'm creating a web crawler which must:
Fetch a web page.
Parse all <a> tags with hrefs on the page.
Classify the tags as either article (Meaning the link ...
0
votes
0
answers
24
views
Classify book titles into categories
I have 10,000 book names sorted into 50 categories. This is my training data.
I want to sort 2,000 or more names to sort with more incoming. What is the simplest way I could do this?
So my training ...
- 109
0
votes
1
answer
51
views
Topic modelling or Keyword extraction for a small dataset
I am working on a project where I have a dataset which contains very less data. These are the comments of people. I have only 130 lines with 10 words per line. My goal is to identify the common topics ...
0
votes
1
answer
48
views
Limitations of NLP BERT model for sentiment analysis
I am reading a paper, where the authors assess online public sentiment in China in response tot the government's policies during Covid-19, using a Chinese BERT model. The author's objective is not ...
- 11
0
votes
1
answer
25
views
LinearSVC training time with CountVectorizer and HashingVectorizer
I am currently trying to build a text classifier and I am experimenting with different settings. Specifically, I am extracting my features with a CountVectorizer ...
- 145
0
votes
0
answers
42
views
NLP Text Classification Model with defined context / intent
This is more of a guideline question rather than a technical query. I am looking to create a classification model that classifies documents based on a specific list of strings. However, as it turns ...
- 101
0
votes
0
answers
7
views
Are there known meanings or topics associated with pretrained word embeddings?
I've been using some pre-trained word embeddings (Glove, FastText, word2vec etc.) in text classification, with the averaged 300 dimensions of tags and sentences as input features. Normally it's useful ...
- 1
0
votes
0
answers
6
views
Combining a BoW vector with other features: effectivity and feature value ranges
I am working with a Support Vector Machine to predict class prevalence in a binary classification problem.
The model will take a sparse representation of an instance as input, where the number of the ...
1
vote
0
answers
8
views
Is It Fundamentally Correct To The Text Classification Model To Train First Without Pre-Trained Word Vectors And Then With Pre-Trained Word Vectors?
Is this solution fundamentally correct to the text classification (sentiment analysis) model to train it by these three steps:
train the model without pre-trained word vectors untill reaches the ...
- 121
1
vote
0
answers
13
views
Is This Solution Fundamentally Correct To The Text Classification Model With Pre-Trained Word Vectors?
Is it fundamentally correct to training text classification (sentiment analysis) model with pre-trained word vectors; first with the locked embedding layer, and then train again with locked additional ...
- 121
3
votes
1
answer
217
views
Performing a text classification based on a dictionary
I have been given a kind of dictionary which maps a category with a set of certain strings. A sample of the dictionary is given below:
This is all I have, there is no other data. There are around 46 ...
1
vote
1
answer
35
views
How to deal with spelling errors in NLP classfier (low resource language)
I know there are questions on how to deal with spelling error NLP - but the question and solution are mainly focused on English where there are tons of library for spell-correction.
Here I am curious ...
- 345
0
votes
0
answers
29
views
Is TF-IDF for text classification transferable between corpuses?
I am using TF-IDF for text classification and my solution works well according to the performance metric of my choice (F1 macro). To speed up the training process I have used PCA to reduce the ...
- 101
0
votes
0
answers
28
views
Batch size to avoid overfitting
I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't ...
- 1
0
votes
0
answers
6
views
Text2Slide multiclass classification
I am considering an idea of stitching together a slide deck based on text input, e.g. given:
...
0
votes
0
answers
19
views
Using a fine-tuned model for a different dataset
I have a dataset of different sentences from news articles which I need to classify by their sentiment. For that goal I'm planning to use a fine-tuned model which was fine-tuned on different datasets, ...
- 1
2
votes
1
answer
81
views
Is there anyway to classify the category on give amazon reviews using python
I am trying to find a model or way to classify text which falls into a category and its a positive or negative feedback.
For ex. we have three columns
Review : Camera's not good battery backup is not ...
1
vote
0
answers
15
views
Contextual word embeddings from pretrained word2vec vectors
I would like to create word embeddings that take context into account, so the vector of the word Jaguar [animal] would be different from the word Jaguar [car brand].
As you know, word2vec only gives ...
- 11
0
votes
0
answers
83
views
Can i use Transformer-XL for text classification task?
I want to use transformer xl for text classification tasks. But I don't know the architect model for the text classification task. I use dense layers with activation softmax for logits output from the ...
- 1
0
votes
0
answers
157
views
Language Detection using pycld2
I am trying to use the pycld2 package to detect multiple languages in text. This package provides Python bindings for the Compact Language Detect 2 (CLD2)
This is the example I am testing out:
...
- 11
0
votes
2
answers
59
views
Suggestions for a multi-class text classification model with a large number of classes?
I was working on a text classification problem where I currently have around 40-45 different labels.
The input is a text sentence with a keyword. For e.g. ...
- 1
0
votes
1
answer
47
views
Ideal Windows Size in Pk Evaluation Metric
I am very new to nlp. I am doing a text segmentation task and for evaluating my model I need to calculate Pk and Windiff scores. My question is what is the ideal value for window size (k) for Pk score ...
- 1
0
votes
0
answers
9
views
Naive Bayes TfidfVectorizer predicts everything to one class
I'm trying to run Multinomial Bayes classificator on various balanced data sets and comparing 2 different vectorizers: TfidfVectorizer and CountVectorizer. I have 3 classes: NEG, NEU and POS. I have ...
2
votes
0
answers
36
views
sentence type classification
I want to classify the sentences in my dataset as declarative, interrogative, imperative and ...
- 33
0
votes
0
answers
6
views
predictive effect in the classification made according to the comments in different fields
I want to do a classification through comments categorized in 4 areas(X,Y,Z,M). Categorizing the product as good or bad based on the comments in the fields X, Y, Z, M. How can I follow a path to see ...
- 9
0
votes
1
answer
21
views
How can I implement text classification for this problem?
Given a collection of documents - each corresponding to some economic entity - I am looking to extract information and populate a table with predetermined headings. I have a small sample of this ...
- 11
0
votes
1
answer
26
views
How to provide Intentional Bias towards recent examples in Text Classification?
I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
2
votes
0
answers
24
views
Multi-Class Document Classification with both known and un-known classes
Currently, I am building a multi-class document classifier which has to classify either 3 known classes, namely "Financial Report", "Insurance_Sheet", "Endorsement", and ...
0
votes
0
answers
39
views
Commercial product name classification with product id
I want to give the probability of how the entered name (user 1) for product X matches with the names (historical names from all users) of product X.
I have data with the following structure:
while ...
0
votes
0
answers
17
views
Deep Regression Ensembles(DRE) - text analysis
I read an article about Deep Regression Ensembles(DRE), which can outperform DNN using SDG.
My question is could I use DRE in text classification? (for example, I can use it instead of LDA) What about ...
- 1
1
vote
1
answer
40
views
How to down\up sample text?
I have data set of 5566 samples - one column is the text of the recipe description and the other is what tax class is it.
I wish to make a classifier that would classify receipts using ML only.
I have ...
- 23
0
votes
0
answers
7
views
Removing footers from text scraped from news sites
I am wondering if anyone knows of libraries out there that will remove footer material from text scraped from news sites, like the material in italics below:
“Go, I’ll catch up with you,” the woman ...
- 101
0
votes
0
answers
9
views
Text segment identification without clear boundaries
I am considering the problem of creating a supervised machine learning algorithm able to identify segments of text that are of interest, but where the segments will not, in general, coincide with a ...
- 247
0
votes
1
answer
32
views
Training a model with a series of text responses as input
I want to train a binary classifier on text -- so something like sentiment analysis, but my input vectors are going to be a series of responses from some user separated by some separator character. I ...
- 133
1
vote
1
answer
365
views
What are the exact differences between Word Embedding and Word Vectorization?
I am learning NLP. I have tried to figure out the exact difference between Word Embedding and Word Vectorization. However, seems like some articles use these words interchangeably. But I think there ...
- 13
2
votes
1
answer
52
views
A multi label text classification problem
I'm looking to solve a multi label text classification problem but I don't really know how to formulate it correctly so I can look it up.. Here is my problem :
Say I have the document ...
- 35
1
vote
1
answer
23
views
Distinguishing text with opposite meanings in SVM (False Information Detection)
I am currently working on a Binary Text Classification Model (False Information Detection) using Support Vector Machine and used TF-IDF as text vectorizer in Python. I have already tried training the ...
- 11