Questions tagged [text-classification]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2
votes
1answer
40 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
0
votes
1answer
24 views

How to handle such a large class imbalance in text data?

I am working on a multi class text classifier. The total number of class that are there is 265 and total number of rows is 20,000. The class with largest number of occurrences has 6000 samples and ...
0
votes
1answer
15 views

Correlation between words, then texts

I have texts collected on different topics. I would like to study the possible correlation between these. I have started looking at the word frequency and it seems that in one dataset the word with ...
0
votes
1answer
29 views

Text classification analysis based on similarity

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
-1
votes
1answer
37 views

How to best handle imbalanced text classification with Keras?

I implemented a text classification model using Keras. Most of the datasets that I use are imbalanced. Therefore, I would like to use SMOTE to handle said imbalance. I tried both on plain text, and ...
0
votes
0answers
11 views

Padding texts before input for Neural Network

I am trying to classify long texts (documents) using neural networks. I am looking for tutorials and code examples in Keras and everything I find pads all texts to the size of the longest one in the ...
0
votes
0answers
19 views

How to build a dictionary for sentiment analysis?

I would like to know if it would be possible to build a dictionary using terms/words included in a dataset, in order to be used for sentiment analysis (and build a sentiment analysis classification ...
0
votes
1answer
20 views

Text classification and predictive model

I have collected a lot of data that I would like to analyse and classified. Unfortunately, they are not already labelled, so I am going to do manually. The dataset consists of texts in Italian and I ...
1
vote
1answer
34 views

How to deal with imbalanced text data

I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance: "Tresemme shampoo and conditioner - sulfate-free" = ...
1
vote
1answer
22 views

Building a text classification model from scratch

I do not know if this is the right place to post my question. I am a beginner in data science and machine learning techniques. I would need to build a modem that can allow me to classify texts and run ...
0
votes
0answers
7 views

Single numeric features row and multiple text rows corresponding to numeric data. How to classify in dataset?

For example, take a review of a product. I have numeric information corresponding to the product and also multiple reviews and I have to output a binary class for the product. How can I predict when I ...
0
votes
1answer
22 views

Text Mapping - Medicine Names

We have a problem where we have a standardized database of Medicine names. On the other hand, there is a subset of medicine names which could have spelling mistakes, different structure or hypens, ...
0
votes
1answer
36 views

Classification - get some label value to check how close to another class (Python)

I am doing text classification in python with 3 alghoritms: kNN, Naive Bayes and SVM. I have 3 classes - easy, medium and hard. The accuracy is quite fine. Is there a way to check for new text its ...
1
vote
1answer
13 views

What are some of the available methods for handling multi-label classification for longer sequences of text

I am looking to solve a multi-class classification problem with long sequences of text with some rows having 1000's of tokens. Some of the state of the art methods such as BERT have a token limit and ...
0
votes
0answers
12 views

Machine Learning - Multilable Text Classification

I am trying to solve a multilable text classification problem and used tf-idf for feature engineering and calibrated+linearSVC into the model. Results are great, however, I am trying to figure out a) ...
0
votes
0answers
24 views

NLP/Text Analytics : How to extract relevant features from a given text so as to predict the future probability of the label assigned?

Actually I am a bit stumped as to how one can approach the problem where for a given historical text data, we have to predict the probability of approval for the new text data. For further ...
0
votes
1answer
23 views

SciKit-Learn: predict values sometimes different from top predict_proba entries?

I noticed that in my text classification problem, I get significantly worse results when I use the max of predict_proba's output as compared to the straight ...
0
votes
1answer
453 views

Overfitting with text classification using Transformers

I am trying to make a binary text classification model by using the encoder part of the transformer and then using its output to feed into an LSTM network. However, I am not able to achieve good ...
0
votes
2answers
72 views

Training a classifier with text and numerical features - what is the state of the art?

I'm trying to build a binary classifier where the features are mostly numerical (about 20) and there are a couple of unstructured short text fields as well. What is currently considered the state of ...
0
votes
0answers
7 views

Can text analysis approaches using machine learning be used on financial statement reports?

In a paper (Correa et al, 2017), the following is mentioned: " Alternative text analysis methods, such as machine-learning approaches, require some type of classification that would help in ...
0
votes
0answers
19 views

Training a cosine similarity matrix for similar text recommendation

I'm working on similar movie names recommender system. I have a dataset of only movie_titles that I converted into matrix using tfidf and then computed the cosine ...
1
vote
0answers
19 views

Detecting low-quality, user-created text content

I run a website that allows visitors to publish text content. It is very similar to a forum in terms of functionality. I'd like to automatically exclude or flag, submitted text content that is "spammy"...
0
votes
3answers
39 views

approach to classify text with natural language processing methods

I have a problem with regards to text classification/categorization. The task is bugging me for days already and as I am pretty new to AI and the field of natural language processing (NLP) I am just ...
1
vote
2answers
28 views

Thematic clustering of text

Please advise on starting points, research (papers,frameworks) related to thematic clustering of text. In particular on a system with two levels of clustering where second level has a temporal nature....
1
vote
2answers
119 views

Text classification into thousands of classes

Could somebody point me to a paper or code that is about classifying texts into potentially thousands of categories (topics)? I do have data based on Wikipedia and the number of categories is really ...
1
vote
1answer
73 views

TF Keras Text Processing - Classification Model

I'm trying to put together a script that classifies comments into either adequate or inadequate. I put a question up here earlier with all my code, but I think I've isolated the problem down into the ...
0
votes
0answers
11 views

ML.net Text Feature Augmentation and Selective Normalisation

I'm in the middle of training a model for multi-class classification, I'm relatively green here and have a few questions for the more seasoned among you. Is it possible to progmatically add a line ...
0
votes
0answers
9 views

Taking huge time to execute piepeline text classification model using sklearn?

I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k ...
2
votes
1answer
19 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
0
votes
1answer
17 views

How to identify new job descriptions/postings from a set of documents when I have a set of already labeled job descriptions/postings

Suppose I have a set of already labeled documents -- some of them are job descriptions/postings (these are documents of interest), and some of them are not. I wonder what kind of method would allow me ...
3
votes
1answer
16 views

Doubt on scope of text classification problem

I have a dataset that describes the sellers who are selling various brands. I need to identify the source (where did he buy those brands he is selling from) of those sellers. (Dimension of dataset 11,...
1
vote
1answer
24 views

Does it make sense to use a tfidf matrix for a model which expects to see new text?

I'm training a model to classify tweets right now. Most of the text classification examples I have seen convert the tweets into tf-idf document term matrices as input for the model. However, this ...
1
vote
1answer
420 views

Using Trainable=True in Keras Embedding obtained better performance

It is suggested by the author of Keras [1] to use Trainable=False when using the embedding layer in Keras to prevent the weights from being updated during training. ...
1
vote
1answer
184 views

Difference between SVM and GD/SGD?

My colleague mentioned that a data science project is using SGD classifier. So I started reading about GD/SGD and came across a nice article about Text classification using SVM and GD. In the end of ...
0
votes
1answer
39 views

How to use text classification where the training source are txt files in categorized folders?

I have 200 *.txt unique files for each folder: Each file is a lawsuit initial text separated by legal areas (folders) of public advocacy. I would like to create a training data to predict new ...
3
votes
1answer
43 views

Classifying one particular class of documents from the rest

I am trying to build a classifier that would classify if a document is a document about sports or not. I have enough samples of sports document to train a classifier on, however I can't imagine how I ...
0
votes
0answers
16 views

Text detection on English and Chinese languages

https://arxiv.org/abs/1910.07954 In this paper, we have a convolutional character neural network where we have object detection by taking a character as a basic unit. First, we do character detection ...
1
vote
0answers
26 views

How to classify unseen text data?

I am training an text classifier for addresses such that if given sentence is an address or not. ...
3
votes
1answer
126 views

How to use ndcg metric for binary relevance

I am working on a ranking problem to predict the right single document based on the user query and use the NDCG metric to measure the model. Given the details : Queries ( Q ), Result Document ( D ),...
6
votes
2answers
3k views

What is purpose of the [CLS] token and why its encoding output is important?

I am reading this article on how to use BERT by Jay Alammar and I understand things up until: For sentence classification, we’re only only interested in BERT’s output for the [CLS] token, so we ...
2
votes
2answers
46 views

Which kind of model is better for keyword-set classification?

There exists a similar task that is named text classification. But I want to find a kind of model that the inputs are keyword set. And the keyword set is not from a sentence. For example: ...

1
2