Questions tagged [text-classification]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
3answers
27 views

Distinguish randomly generated texts from reasonable for human texts

I have strings short texts of 2 types: '23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ... and ...
0
votes
1answer
32 views

How can a ML algorithm learn to classify fake news? [duplicate]

I am new in Machine learning techniques and in fake news detection by using these algorithms (SVM, nn, logistic regression,..). I would like to understand how an algorithm can learn from a training ...
0
votes
0answers
13 views

Classify Spanish Text into different Categories

I want to recommend articles to users depending upon what type of article is user reading, Music, Movies, Politics, etc. I have 3 features: Page Title, Labels, article content. I am using an API (...
0
votes
1answer
28 views

Ordering of standardization, pca, and/or tfidf for neural network

I have 60k rows of text data. I have tokenized it into 55k columns. I am using a neural network to classify the data but have some questions about how to order my preprocessing steps. I have too much ...
0
votes
0answers
14 views

What should an algorithm for detecting fake news include? [closed]

I do not know if anyone of you has thought about fake news or worked on detection, but I was wondering what a good algorithm for detecting fake news should take into account to be able to determine if ...
0
votes
0answers
11 views

Trying to use CBOW for tweet classification

I'm trying to use the Continuous Bag Of Words method for word embedding on a corpus of 7503 tweets. In particular, I'm trying to use CBOW on this Kaggle competition, which involves classifying tweets ...
3
votes
1answer
81 views

How to include categorical fields to enhance a text classification

I would have a question on how to add more categorical fields in a classification problem. My dataset had initially 4 fields: ...
0
votes
1answer
19 views

Is it possible to classify documents of corpus using labels?

I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
0
votes
1answer
16 views

How to categorize unlabelled promotional email data

I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
2
votes
1answer
19 views

Interpreting confidence interval results for datasets

I have created a dataset automatically and wanted to clarify my interpretation of the amount of noise using the confidence interval. I selected a random sample and manually annotated the sample and ...
0
votes
1answer
36 views

Classifiers and accuracy

I would like to ask you how to use classifier and determine accuracy of models. I have my dataset and I already cleaned the text (remove stopwords, punctuation, removed empty rows,...). Then I split ...
0
votes
0answers
16 views

How to apply an algorithm for text classification?

I would like to know more about algorithms to apply for text classifications. I know that steps include also remove of punctuation and stop words and tokenisation to clean the text. If I already ...
0
votes
1answer
19 views

How to create a Document Categorization Classifier for different contexts of Documents

I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities. The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
1
vote
2answers
24 views

Problem of continuous training - Supervised learning

I am sure this is a most common problem, but would like to know by experts on how to tackle it. Note that, I mostly deal with textual data (NLP problems). When a supervised learning model is created, ...
1
vote
1answer
38 views

Creating a valid dataset for obtaining results

I have created a domain-specific dataset, lets say it is relating to python programming topic posts. I have taken data from various places specific to this topic to create positive examples in my ...
0
votes
0answers
15 views

How to approach a text parsing problem

I'm looking to parse human-generated cyber-physical sensor tags into a series of standardised tags and identifiers. For example "Rm3.ZnT-SP" refers to ROOM ID:3 ZONE TEMPERATURE SETPOINT The ...
2
votes
1answer
36 views

Any useful tips on transfer learning for a text classification task

I am doing a supervised binary text classification task. I want to classify the texts from site A, site B, and site C. The in-domain performance looks OK for texts of each site. (92%-94% accuracy). ...
0
votes
1answer
20 views

(Serious) Dataset of paedophilic Youtube comments (or similar)? [closed]

I think it would be useful to create a model that tries to predict whether a youtube comment is paedophilic - maybe the model should also take into account the channel name/description/front image. It'...
1
vote
1answer
37 views

Text classification with Word2Vec on a larger corpus

I am working on a small project and I would like to use the word2vec technique as a text representation method. I need to classify patents but I have only a few of them labelled and to increase the ...
1
vote
1answer
27 views

Identifying templates from SMS text

I am building an app where I identify information from the SMS, something similar to expense management apps. I have a parser which reads all the SMS of user, identifies SMS of interest and parses ...
0
votes
1answer
41 views

Sklearn SVM question classification

So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset: The goal of this task is to ...
0
votes
0answers
43 views

Hierarchical transformer for document classification: error with model implementation, and advice on extracting attention weights

I am trying to implement a hierarchical transformer for document classification in Keras/tensorflow, in which: (1) a word-level transformer produces a representation of each sentence, and attention ...
2
votes
1answer
39 views

Is there any way to plot ROC curve for Ensemble hard voting classifier?

I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
0
votes
0answers
17 views

Supervised Text Classification

I need to create a summarized report showing how many containers are being imported by each company. This report needs to be generated on monthly basis. The company names are entered in a way that ...
2
votes
2answers
25 views

How to identify text similarity based on training data?

I have a set of documents (1 to 11) for which the labeling is done. Lets Assume: ...
0
votes
0answers
24 views

Algorithm for rectifying incorrect text from OCR

I have an OCR (tesseract) engine running on processing "chyron" text from a live stream news feed. However, the OCR is often inaccurate and ends up with some noisy words: ...
0
votes
0answers
25 views

use genetic algorithm as a feature selection for text classification

how to apply the genetic algorithm as a feature selection for text classification in python I need to use GA to select most relevant feature in text classification
3
votes
0answers
33 views

Bag of words: Prediction on new (out-of-sample) data

I'm working with a bag of words in R: library(tm) corpus = VCorpus(textsource) dtm = DocumentTermMatrix(corpus) dtm = as.matrix(dtm) I use the matrix ...
2
votes
2answers
38 views

Text Classification : Classifying N classes vs rest of the classes

Apologies if this is naive, I am fairly new to the domain. I have a requirement where I am trying to classify 2 types of text data, i.e, I have got 2 classes to classify my data upon. I am able to get ...
0
votes
1answer
23 views

K-means and LDA for text classification

I hope to explain in a clear way what I would like to do. I have more than 50000 tweets and I would like to add some labels on topics. So I have used LDA for doing this. I have also used k-means to ...
0
votes
2answers
29 views

k-means and LDA for text classification: how to test accuracy?

I have many tweets that I would like classify based on their similarity. Unfortunately I am not quite familiar with text-classification and nlp, so I had to read a lot of documents before having an ...
1
vote
2answers
42 views

How many layers should I replace in transfer learning CNN

I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
1
vote
2answers
25 views

Confusion with using different classes in neural networks (training vs testing)

I am new to deep learning and I am confused about having a neural network trained on certain classes and tested on different ones. Suppose I want to have a convolutional neural network that learns ...
2
votes
1answer
27 views

Classify text as logical/ not logical

Can some one advise me direction where to look in.Or some resources. Here is a task: User leaves feed back-text with min 50 characters. I need to check if it's normal human sentences/ word ...
0
votes
2answers
16 views

Need for Dense layer in Text Classifcation

While creating a model for text classification, what is the need for a Dense Layer? I noticed in multiple examples the following is the structure. A softmax is what required right instead of the Dense ...
0
votes
1answer
20 views

Text Classification on a very small data set with a lot of classes

I have a data set consisting of 455 rows spread over 21 different classes. The data set is imbalanced as well as you can see below. ...
0
votes
4answers
56 views

Clusters: how to improve results for text classification

I am trying to classify texts using kmeans, TfidfVectorizer, PCA. However, it seems that many texts are not correctly classified as you can see: I have texts in cluster2 that should be in Cluster 0 or ...
0
votes
0answers
27 views

How to properly vectorise when I have several text features?

I am having several features as text and I'm using them once for a classification problem and then later for a regression problem. The textual data features themselves aren't categorical. Because as ...
1
vote
1answer
33 views

Which insights a data scientist could derive from text-analysis? [closed]

I have many texts and I am trying to analyse them. After tokenising them, studying words frequency, spotting any typos, studying punctuations, I have been working on POS tagging. Since it is my first ...
0
votes
0answers
23 views

Text Classification One-shot learning (1sample/class ~1000 classes)

I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000). ...
2
votes
1answer
23 views

How to use ontologies for text classification?

I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset). My question is how ...
0
votes
0answers
9 views

Dirichlet smoothing as an IDF component

How can Dirichlet smoothing be used as an IDF component to estimate the probabilities of a Topic model ? i.e, Smoothing with a background collection model to estimate topic model ? I've seen many ...
0
votes
1answer
26 views

Target in a dataset for text classification

I have a dataset that lacks the target variable. The dataset contains a list of emails and I would need to classify them as spam/not spam. Do you know how I can build a model that can help me to ...
0
votes
0answers
15 views

Calculating accuracy in Extractive Summarization

I am trying to implement a text summarization model. I am using keras tensorflow anad I have used bert sentence embeddings and the output of the embeddings are feeded into an LSTM and then to a Dense ...
0
votes
0answers
26 views

How to label data to build a model from scratch

I have different texts, some of them have been manually labelled some others have not. Texts “The modelling dataset were rebuilt yesterday” “This movie is awful” “Trump said that ufo exist” I would ...
1
vote
0answers
22 views

Spam/ham classification

I am exploring the use of lime for spam/ham categorisation. specifically I have a data frame having list of messages. I would need to identify which messages are spam and which ones are ham by using a ...
1
vote
1answer
20 views

Text classification: accuracy [closed]

I would like to understand how to compute the accuracy of cluster analysis. I have hundred of texts. The dataset looks like as follows: ...
0
votes
1answer
16 views

text classification : comparing classification reports

I have a 4-leabelled text classification problem. Could someone help me choose among the below text classifiers ? I was advised to select the second one ( the one which uses both unigrams and ...
0
votes
0answers
8 views

How to classify very short text for spend analytics?

I have a very small description of the item, example below- ACCELATOR SHAFT ADAPTOR –EGR PIPE & EGR VALVE(125KVA) ADAPTOR-EGR COOLER AND VALVE-125 KVA ADJUSTING LATCH - ALTERNATOR-125 KVA Some ...