Data Science

Questions tagged [text-classification]

Ask Question

The tag has no usage guidance.

52 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
1
vote
1answer
14 views

Which insights a data scientist could derive from text-analysis?

I have many texts and I am trying to analyse them. After tokenising them, studying words frequency, spotting any typos, studying punctuations, I have been working on POS tagging. Since it is my first ...
0
votes
0answers
9 views

Text Classification One-shot learning (1sample/class ~1000 classes)

I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000). ...
1
vote
0answers
12 views

How to use ontologies for text classification?

I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset). My question is how ...
0
votes
0answers
4 views

Dirichlet smoothing as an IDF component

How can Dirichlet smoothing be used as an IDF component to estimate the probabilities of a Topic model ? i.e, Smoothing with a background collection model to estimate topic model ? I've seen many ...
0
votes
1answer
15 views

Target in a dataset for text classification

I have a dataset that lacks the target variable. The dataset contains a list of emails and I would need to classify them as spam/not spam. Do you know how I can build a model that can help me to ...
0
votes
0answers
13 views

Calculating accuracy in Extractive Summarization

I am trying to implement a text summarization model. I am using keras tensorflow anad I have used bert sentence embeddings and the output of the embeddings are feeded into an LSTM and then to a Dense ...
0
votes
0answers
20 views

How to label data to build a model from scratch

I have different texts, some of them have been manually labelled some others have not. Texts “The modelling dataset were rebuilt yesterday” “This movie is awful” “Trump said that ufo exist” I would ...
1
vote
0answers
17 views

Spam/ham classification

I am exploring the use of lime for spam/ham categorisation. specifically I have a data frame having list of messages. I would need to identify which messages are spam and which ones are ham by using a ...
1
vote
1answer
18 views

Text classification: accuracy [closed]

I would like to understand how to compute the accuracy of cluster analysis. I have hundred of texts. The dataset looks like as follows: ...
0
votes
1answer
15 views

text classification : comparing classification reports

I have a 4-leabelled text classification problem. Could someone help me choose among the below text classifiers ? I was advised to select the second one ( the one which uses both unigrams and ...
0
votes
0answers
7 views

How to classify very short text for spend analytics?

I have a very small description of the item, example below- ACCELATOR SHAFT ADAPTOR –EGR PIPE & EGR VALVE(125KVA) ADAPTOR-EGR COOLER AND VALVE-125 KVA ADJUSTING LATCH - ALTERNATOR-125 KVA Some ...
2
votes
1answer
24 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
0
votes
1answer
16 views

How to handle such a large class imbalance in text data?

I am working on a multi class text classifier. The total number of class that are there is 265 and total number of rows is 20,000. The class with largest number of occurrences has 6000 samples and ...
0
votes
1answer
14 views

Correlation between words, then texts

I have texts collected on different topics. I would like to study the possible correlation between these. I have started looking at the word frequency and it seems that in one dataset the word with ...
0
votes
1answer
19 views

Text classification analysis based on similarity

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
-1
votes
1answer
20 views

How to best handle imbalanced text classification with Keras?

I implemented a text classification model using Keras. Most of the datasets that I use are imbalanced. Therefore, I would like to use SMOTE to handle said imbalance. I tried both on plain text, and ...
0
votes
0answers
10 views

Padding texts before input for Neural Network

I am trying to classify long texts (documents) using neural networks. I am looking for tutorials and code examples in Keras and everything I find pads all texts to the size of the longest one in the ...
0
votes
0answers
17 views

How to build a dictionary for sentiment analysis?

I would like to know if it would be possible to build a dictionary using terms/words included in a dataset, in order to be used for sentiment analysis (and build a sentiment analysis classification ...
0
votes
1answer
18 views

Text classification and predictive model

I have collected a lot of data that I would like to analyse and classified. Unfortunately, they are not already labelled, so I am going to do manually. The dataset consists of texts in Italian and I ...
1
vote
1answer
31 views

How to deal with imbalanced text data

I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance: "Tresemme shampoo and conditioner - sulfate-free" = ...
0
votes
1answer
17 views

Building a text classification model from scratch

I do not know if this is the right place to post my question. I am a beginner in data science and machine learning techniques. I would need to build a modem that can allow me to classify texts and run ...
0
votes
0answers
7 views

Single numeric features row and multiple text rows corresponding to numeric data. How to classify in dataset?

For example, take a review of a product. I have numeric information corresponding to the product and also multiple reviews and I have to output a binary class for the product. How can I predict when I ...
0
votes
1answer
20 views

Text Mapping - Medicine Names

We have a problem where we have a standardized database of Medicine names. On the other hand, there is a subset of medicine names which could have spelling mistakes, different structure or hypens, ...
0
votes
1answer
35 views

Classification - get some label value to check how close to another class (Python)

I am doing text classification in python with 3 alghoritms: kNN, Naive Bayes and SVM. I have 3 classes - easy, medium and hard. The accuracy is quite fine. Is there a way to check for new text its ...
1
vote
1answer
9 views

What are some of the available methods for handling multi-label classification for longer sequences of text

I am looking to solve a multi-class classification problem with long sequences of text with some rows having 1000's of tokens. Some of the state of the art methods such as BERT have a token limit and ...
0
votes
0answers
10 views

Machine Learning - Multilable Text Classification

I am trying to solve a multilable text classification problem and used tf-idf for feature engineering and calibrated+linearSVC into the model. Results are great, however, I am trying to figure out a) ...
0
votes
0answers
22 views

NLP/Text Analytics : How to extract relevant features from a given text so as to predict the future probability of the label assigned?

Actually I am a bit stumped as to how one can approach the problem where for a given historical text data, we have to predict the probability of approval for the new text data. For further ...
0
votes
1answer
11 views

SciKit-Learn: predict values sometimes different from top predict_proba entries?

I noticed that in my text classification problem, I get significantly worse results when I use the max of predict_proba's output as compared to the straight ...
1
vote
1answer
85 views

Overfitting with text classification using Transformers

I am trying to make a binary text classification model by using the encoder part of the transformer and then using its output to feed into an LSTM network. However, I am not able to achieve good ...
0
votes
2answers
32 views

Training a classifier with text and numerical features - what is the state of the art?

I'm trying to build a binary classifier where the features are mostly numerical (about 20) and there are a couple of unstructured short text fields as well. What is currently considered the state of ...
0
votes
0answers
4 views

Can text analysis approaches using machine learning be used on financial statement reports?

In a paper (Correa et al, 2017), the following is mentioned: " Alternative text analysis methods, such as machine-learning approaches, require some type of classification that would help in ...
0
votes
0answers
6 views

Training a cosine similarity matrix for similar text recommendation

I'm working on similar movie names recommender system. I have a dataset of only movie_titles that I converted into matrix using tfidf and then computed the cosine ...
1
vote
0answers
17 views

Detecting low-quality, user-created text content

I run a website that allows visitors to publish text content. It is very similar to a forum in terms of functionality. I'd like to automatically exclude or flag, submitted text content that is "spammy"...
0
votes
3answers
23 views

approach to classify text with natural language processing methods

I have a problem with regards to text classification/categorization. The task is bugging me for days already and as I am pretty new to AI and the field of natural language processing (NLP) I am just ...
0
votes
2answers
23 views

Thematic clustering of text

Please advise on starting points, research (papers,frameworks) related to thematic clustering of text. In particular on a system with two levels of clustering where second level has a temporal nature....
1
vote
2answers
50 views

Text classification into thousands of classes

Could somebody point me to a paper or code that is about classifying texts into potentially thousands of categories (topics)? I do have data based on Wikipedia and the number of categories is really ...
1
vote
1answer
53 views

TF Keras Text Processing - Classification Model

I'm trying to put together a script that classifies comments into either adequate or inadequate. I put a question up here earlier with all my code, but I think I've isolated the problem down into the ...
0
votes
0answers
9 views

ML.net Text Feature Augmentation and Selective Normalisation

I'm in the middle of training a model for multi-class classification, I'm relatively green here and have a few questions for the more seasoned among you. Is it possible to progmatically add a line ...
0
votes
0answers
7 views

Taking huge time to execute piepeline text classification model using sklearn?

I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k ...
2
votes
1answer
16 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
0
votes
1answer
13 views

How to identify new job descriptions/postings from a set of documents when I have a set of already labeled job descriptions/postings

Suppose I have a set of already labeled documents -- some of them are job descriptions/postings (these are documents of interest), and some of them are not. I wonder what kind of method would allow me ...
2
votes
1answer
13 views

Doubt on scope of text classification problem

I have a dataset that describes the sellers who are selling various brands. I need to identify the source (where did he buy those brands he is selling from) of those sellers. (Dimension of dataset 11,...
1
vote
1answer
18 views

Does it make sense to use a tfidf matrix for a model which expects to see new text?

I'm training a model to classify tweets right now. Most of the text classification examples I have seen convert the tweets into tf-idf document term matrices as input for the model. However, this ...
0
votes
1answer
192 views

Using Trainable=True in Keras Embedding obtained better performance

It is suggested by the author of Keras [1] to use Trainable=False when using the embedding layer in Keras to prevent the weights from being updated during training. ...
1
vote
1answer
101 views

Difference between SVM and GD/SGD?

My colleague mentioned that a data science project is using SGD classifier. So I started reading about GD/SGD and came across a nice article about Text classification using SVM and GD. In the end of ...
0
votes
1answer
26 views

How to use text classification where the training source are txt files in categorized folders?

I have 200 *.txt unique files for each folder: Each file is a lawsuit initial text separated by legal areas (folders) of public advocacy. I would like to create a training data to predict new ...
2
votes
1answer
37 views

Classifying one particular class of documents from the rest

I am trying to build a classifier that would classify if a document is a document about sports or not. I have enough samples of sports document to train a classifier on, however I can't imagine how I ...
0
votes
0answers
16 views

Text detection on English and Chinese languages

https://arxiv.org/abs/1910.07954 In this paper, we have a convolutional character neural network where we have object detection by taking a character as a basic unit. First, we do character detection ...
1
vote
0answers
22 views

How to classify unseen text data?

I am training an text classifier for addresses such that if given sentence is an address or not. ...
2
votes
0answers
78 views

How to use ndcg metric for binary relevance

I am working on a ranking problem to predict the right single document based on the user query and use the NDCG metric to measure the model. Given the details : Queries ( Q ), Result Document ( D ),...

1
2