Questions tagged [text-classification]

The tag has no usage guidance.

50 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
35 views

Bag of words: Prediction on new (out-of-sample) data

I'm working with a bag of words in R: library(tm) corpus = VCorpus(textsource) dtm = DocumentTermMatrix(corpus) dtm = as.matrix(dtm) I use the matrix ...
2
votes
1answer
44 views

Is there any way to plot ROC curve for Ensemble hard voting classifier?

I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
2
votes
2answers
26 views

How to identify text similarity based on training data?

I have a set of documents (1 to 11) for which the labeling is done. Lets Assume: ...
2
votes
1answer
23 views

How to use ontologies for text classification?

I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset). My question is how ...
2
votes
1answer
43 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
2
votes
1answer
19 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
1
vote
2answers
26 views

Confusion with using different classes in neural networks (training vs testing)

I am new to deep learning and I am confused about having a neural network trained on certain classes and tested on different ones. Suppose I want to have a convolutional neural network that learns ...
1
vote
0answers
22 views

Spam/ham classification

I am exploring the use of lime for spam/ham categorisation. specifically I have a data frame having list of messages. I would need to identify which messages are spam and which ones are ham by using a ...
1
vote
1answer
34 views

How to deal with imbalanced text data

I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance: "Tresemme shampoo and conditioner - sulfate-free" = ...
1
vote
1answer
22 views

Building a text classification model from scratch

I do not know if this is the right place to post my question. I am a beginner in data science and machine learning techniques. I would need to build a modem that can allow me to classify texts and run ...
1
vote
1answer
13 views

What are some of the available methods for handling multi-label classification for longer sequences of text

I am looking to solve a multi-class classification problem with long sequences of text with some rows having 1000's of tokens. Some of the state of the art methods such as BERT have a token limit and ...
1
vote
0answers
19 views

Detecting low-quality, user-created text content

I run a website that allows visitors to publish text content. It is very similar to a forum in terms of functionality. I'd like to automatically exclude or flag, submitted text content that is "spammy"...
1
vote
1answer
76 views

TF Keras Text Processing - Classification Model

I'm trying to put together a script that classifies comments into either adequate or inadequate. I put a question up here earlier with all my code, but I think I've isolated the problem down into the ...
1
vote
0answers
26 views

How to classify unseen text data?

I am training an text classifier for addresses such that if given sentence is an address or not. ...
0
votes
0answers
13 views

Classify Spanish Text into different Categories

I want to recommend articles to users depending upon what type of article is user reading, Music, Movies, Politics, etc. I have 3 features: Page Title, Labels, article content. I am using an API (...
0
votes
0answers
11 views

Trying to use CBOW for tweet classification

I'm trying to use the Continuous Bag Of Words method for word embedding on a corpus of 7503 tweets. In particular, I'm trying to use CBOW on this Kaggle competition, which involves classifying tweets ...
0
votes
1answer
19 views

Is it possible to classify documents of corpus using labels?

I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
0
votes
1answer
21 views

How to categorize unlabelled promotional email data

I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
0
votes
0answers
16 views

How to apply an algorithm for text classification?

I would like to know more about algorithms to apply for text classifications. I know that steps include also remove of punctuation and stop words and tokenisation to clean the text. If I already ...
0
votes
0answers
15 views

How to approach a text parsing problem

I'm looking to parse human-generated cyber-physical sensor tags into a series of standardised tags and identifiers. For example "Rm3.ZnT-SP" refers to ROOM ID:3 ZONE TEMPERATURE SETPOINT The ...
0
votes
1answer
41 views

Sklearn SVM question classification

So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset: The goal of this task is to ...
0
votes
0answers
45 views

Hierarchical transformer for document classification: error with model implementation, and advice on extracting attention weights

I am trying to implement a hierarchical transformer for document classification in Keras/tensorflow, in which: (1) a word-level transformer produces a representation of each sentence, and attention ...
0
votes
0answers
17 views

Supervised Text Classification

I need to create a summarized report showing how many containers are being imported by each company. This report needs to be generated on monthly basis. The company names are entered in a way that ...
0
votes
0answers
24 views

Algorithm for rectifying incorrect text from OCR

I have an OCR (tesseract) engine running on processing "chyron" text from a live stream news feed. However, the OCR is often inaccurate and ends up with some noisy words: ...
0
votes
0answers
26 views

use genetic algorithm as a feature selection for text classification

how to apply the genetic algorithm as a feature selection for text classification in python I need to use GA to select most relevant feature in text classification
0
votes
1answer
23 views

K-means and LDA for text classification

I hope to explain in a clear way what I would like to do. I have more than 50000 tweets and I would like to add some labels on topics. So I have used LDA for doing this. I have also used k-means to ...
0
votes
0answers
27 views

How to properly vectorise when I have several text features?

I am having several features as text and I'm using them once for a classification problem and then later for a regression problem. The textual data features themselves aren't categorical. Because as ...
0
votes
0answers
23 views

Text Classification One-shot learning (1sample/class ~1000 classes)

I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000). ...
0
votes
0answers
9 views

Dirichlet smoothing as an IDF component

How can Dirichlet smoothing be used as an IDF component to estimate the probabilities of a Topic model ? i.e, Smoothing with a background collection model to estimate topic model ? I've seen many ...
0
votes
1answer
26 views

Target in a dataset for text classification

I have a dataset that lacks the target variable. The dataset contains a list of emails and I would need to classify them as spam/not spam. Do you know how I can build a model that can help me to ...
0
votes
0answers
15 views

Calculating accuracy in Extractive Summarization

I am trying to implement a text summarization model. I am using keras tensorflow anad I have used bert sentence embeddings and the output of the embeddings are feeded into an LSTM and then to a Dense ...
0
votes
0answers
26 views

How to label data to build a model from scratch

I have different texts, some of them have been manually labelled some others have not. Texts “The modelling dataset were rebuilt yesterday” “This movie is awful” “Trump said that ufo exist” I would ...
0
votes
0answers
8 views

How to classify very short text for spend analytics?

I have a very small description of the item, example below- ACCELATOR SHAFT ADAPTOR –EGR PIPE & EGR VALVE(125KVA) ADAPTOR-EGR COOLER AND VALVE-125 KVA ADJUSTING LATCH - ALTERNATOR-125 KVA Some ...
0
votes
1answer
29 views

Text classification analysis based on similarity

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
0
votes
0answers
11 views

Padding texts before input for Neural Network

I am trying to classify long texts (documents) using neural networks. I am looking for tutorials and code examples in Keras and everything I find pads all texts to the size of the longest one in the ...
0
votes
0answers
19 views

How to build a dictionary for sentiment analysis?

I would like to know if it would be possible to build a dictionary using terms/words included in a dataset, in order to be used for sentiment analysis (and build a sentiment analysis classification ...
0
votes
1answer
20 views

Text classification and predictive model

I have collected a lot of data that I would like to analyse and classified. Unfortunately, they are not already labelled, so I am going to do manually. The dataset consists of texts in Italian and I ...
0
votes
0answers
7 views

Single numeric features row and multiple text rows corresponding to numeric data. How to classify in dataset?

For example, take a review of a product. I have numeric information corresponding to the product and also multiple reviews and I have to output a binary class for the product. How can I predict when I ...
0
votes
1answer
22 views

Text Mapping - Medicine Names

We have a problem where we have a standardized database of Medicine names. On the other hand, there is a subset of medicine names which could have spelling mistakes, different structure or hypens, ...
0
votes
1answer
36 views

Classification - get some label value to check how close to another class (Python)

I am doing text classification in python with 3 alghoritms: kNN, Naive Bayes and SVM. I have 3 classes - easy, medium and hard. The accuracy is quite fine. Is there a way to check for new text its ...
0
votes
0answers
12 views

Machine Learning - Multilable Text Classification

I am trying to solve a multilable text classification problem and used tf-idf for feature engineering and calibrated+linearSVC into the model. Results are great, however, I am trying to figure out a) ...
0
votes
0answers
24 views

NLP/Text Analytics : How to extract relevant features from a given text so as to predict the future probability of the label assigned?

Actually I am a bit stumped as to how one can approach the problem where for a given historical text data, we have to predict the probability of approval for the new text data. For further ...
0
votes
0answers
7 views

Can text analysis approaches using machine learning be used on financial statement reports?

In a paper (Correa et al, 2017), the following is mentioned: " Alternative text analysis methods, such as machine-learning approaches, require some type of classification that would help in ...
0
votes
0answers
19 views

Training a cosine similarity matrix for similar text recommendation

I'm working on similar movie names recommender system. I have a dataset of only movie_titles that I converted into matrix using tfidf and then computed the cosine ...
0
votes
3answers
39 views

approach to classify text with natural language processing methods

I have a problem with regards to text classification/categorization. The task is bugging me for days already and as I am pretty new to AI and the field of natural language processing (NLP) I am just ...
0
votes
0answers
11 views

ML.net Text Feature Augmentation and Selective Normalisation

I'm in the middle of training a model for multi-class classification, I'm relatively green here and have a few questions for the more seasoned among you. Is it possible to progmatically add a line ...
0
votes
0answers
9 views

Taking huge time to execute piepeline text classification model using sklearn?

I have created a pipeline model for text classification using python ,Firstly i have tried on 30k records dataset it is working fine got the good results , but when it comes to huge data set like 50k ...
0
votes
1answer
17 views

How to identify new job descriptions/postings from a set of documents when I have a set of already labeled job descriptions/postings

Suppose I have a set of already labeled documents -- some of them are job descriptions/postings (these are documents of interest), and some of them are not. I wonder what kind of method would allow me ...
0
votes
1answer
40 views

How to use text classification where the training source are txt files in categorized folders?

I have 200 *.txt unique files for each folder: Each file is a lawsuit initial text separated by legal areas (folders) of public advocacy. I would like to create a training data to predict new ...
0
votes
0answers
16 views

Text detection on English and Chinese languages

https://arxiv.org/abs/1910.07954 In this paper, we have a convolutional character neural network where we have object detection by taking a character as a basic unit. First, we do character detection ...