Questions tagged [text-classification]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
3answers
27 views

Distinguish randomly generated texts from reasonable for human texts

I have strings short texts of 2 types: '23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ... and ...
0
votes
3answers
39 views

approach to classify text with natural language processing methods

I have a problem with regards to text classification/categorization. The task is bugging me for days already and as I am pretty new to AI and the field of natural language processing (NLP) I am just ...
0
votes
1answer
32 views

How can a ML algorithm learn to classify fake news? [duplicate]

I am new in Machine learning techniques and in fake news detection by using these algorithms (SVM, nn, logistic regression,..). I would like to understand how an algorithm can learn from a training ...
3
votes
1answer
126 views

How to use ndcg metric for binary relevance

I am working on a ranking problem to predict the right single document based on the user query and use the NDCG metric to measure the model. Given the details : Queries ( Q ), Result Document ( D ),...
0
votes
1answer
28 views

Ordering of standardization, pca, and/or tfidf for neural network

I have 60k rows of text data. I have tokenized it into 55k columns. I am using a neural network to classify the data but have some questions about how to order my preprocessing steps. I have too much ...
0
votes
0answers
13 views

Classify Spanish Text into different Categories

I want to recommend articles to users depending upon what type of article is user reading, Music, Movies, Politics, etc. I have 3 features: Page Title, Labels, article content. I am using an API (...
0
votes
0answers
14 views

What should an algorithm for detecting fake news include? [closed]

I do not know if anyone of you has thought about fake news or worked on detection, but I was wondering what a good algorithm for detecting fake news should take into account to be able to determine if ...
3
votes
1answer
81 views

How to include categorical fields to enhance a text classification

I would have a question on how to add more categorical fields in a classification problem. My dataset had initially 4 fields: ...
0
votes
0answers
11 views

Trying to use CBOW for tweet classification

I'm trying to use the Continuous Bag Of Words method for word embedding on a corpus of 7503 tweets. In particular, I'm trying to use CBOW on this Kaggle competition, which involves classifying tweets ...
1
vote
1answer
73 views

TF Keras Text Processing - Classification Model

I'm trying to put together a script that classifies comments into either adequate or inadequate. I put a question up here earlier with all my code, but I think I've isolated the problem down into the ...
2
votes
1answer
19 views

Interpreting confidence interval results for datasets

I have created a dataset automatically and wanted to clarify my interpretation of the amount of noise using the confidence interval. I selected a random sample and manually annotated the sample and ...
0
votes
1answer
19 views

Is it possible to classify documents of corpus using labels?

I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
0
votes
1answer
16 views

How to categorize unlabelled promotional email data

I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
2
votes
1answer
19 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
2
votes
2answers
25 views

How to identify text similarity based on training data?

I have a set of documents (1 to 11) for which the labeling is done. Lets Assume: ...
0
votes
0answers
23 views

Text Classification One-shot learning (1sample/class ~1000 classes)

I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000). ...
0
votes
1answer
36 views

Classifiers and accuracy

I would like to ask you how to use classifier and determine accuracy of models. I have my dataset and I already cleaned the text (remove stopwords, punctuation, removed empty rows,...). Then I split ...
0
votes
1answer
19 views

How to create a Document Categorization Classifier for different contexts of Documents

I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities. The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
0
votes
1answer
41 views

Sklearn SVM question classification

So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset: The goal of this task is to ...
2
votes
1answer
39 views

Is there any way to plot ROC curve for Ensemble hard voting classifier?

I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
0
votes
0answers
16 views

How to apply an algorithm for text classification?

I would like to know more about algorithms to apply for text classifications. I know that steps include also remove of punctuation and stop words and tokenisation to clean the text. If I already ...
0
votes
1answer
17 views

How to identify new job descriptions/postings from a set of documents when I have a set of already labeled job descriptions/postings

Suppose I have a set of already labeled documents -- some of them are job descriptions/postings (these are documents of interest), and some of them are not. I wonder what kind of method would allow me ...
1
vote
2answers
24 views

Problem of continuous training - Supervised learning

I am sure this is a most common problem, but would like to know by experts on how to tackle it. Note that, I mostly deal with textual data (NLP problems). When a supervised learning model is created, ...
1
vote
1answer
38 views

Creating a valid dataset for obtaining results

I have created a domain-specific dataset, lets say it is relating to python programming topic posts. I have taken data from various places specific to this topic to create positive examples in my ...
0
votes
0answers
15 views

How to approach a text parsing problem

I'm looking to parse human-generated cyber-physical sensor tags into a series of standardised tags and identifiers. For example "Rm3.ZnT-SP" refers to ROOM ID:3 ZONE TEMPERATURE SETPOINT The ...
2
votes
1answer
36 views

Any useful tips on transfer learning for a text classification task

I am doing a supervised binary text classification task. I want to classify the texts from site A, site B, and site C. The in-domain performance looks OK for texts of each site. (92%-94% accuracy). ...
1
vote
2answers
25 views

Confusion with using different classes in neural networks (training vs testing)

I am new to deep learning and I am confused about having a neural network trained on certain classes and tested on different ones. Suppose I want to have a convolutional neural network that learns ...
0
votes
1answer
23 views

K-means and LDA for text classification

I hope to explain in a clear way what I would like to do. I have more than 50000 tweets and I would like to add some labels on topics. So I have used LDA for doing this. I have also used k-means to ...
0
votes
1answer
20 views

(Serious) Dataset of paedophilic Youtube comments (or similar)? [closed]

I think it would be useful to create a model that tries to predict whether a youtube comment is paedophilic - maybe the model should also take into account the channel name/description/front image. It'...
1
vote
1answer
37 views

Text classification with Word2Vec on a larger corpus

I am working on a small project and I would like to use the word2vec technique as a text representation method. I need to classify patents but I have only a few of them labelled and to increase the ...
2
votes
1answer
23 views

How to use ontologies for text classification?

I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset). My question is how ...
1
vote
1answer
27 views

Identifying templates from SMS text

I am building an app where I identify information from the SMS, something similar to expense management apps. I have a parser which reads all the SMS of user, identifies SMS of interest and parses ...
0
votes
0answers
43 views

Hierarchical transformer for document classification: error with model implementation, and advice on extracting attention weights

I am trying to implement a hierarchical transformer for document classification in Keras/tensorflow, in which: (1) a word-level transformer produces a representation of each sentence, and attention ...
0
votes
0answers
17 views

Supervised Text Classification

I need to create a summarized report showing how many containers are being imported by each company. This report needs to be generated on monthly basis. The company names are entered in a way that ...
0
votes
0answers
24 views

Algorithm for rectifying incorrect text from OCR

I have an OCR (tesseract) engine running on processing "chyron" text from a live stream news feed. However, the OCR is often inaccurate and ends up with some noisy words: ...
0
votes
0answers
25 views

use genetic algorithm as a feature selection for text classification

how to apply the genetic algorithm as a feature selection for text classification in python I need to use GA to select most relevant feature in text classification
0
votes
1answer
39 views

How to use text classification where the training source are txt files in categorized folders?

I have 200 *.txt unique files for each folder: Each file is a lawsuit initial text separated by legal areas (folders) of public advocacy. I would like to create a training data to predict new ...
3
votes
1answer
43 views

Classifying one particular class of documents from the rest

I am trying to build a classifier that would classify if a document is a document about sports or not. I have enough samples of sports document to train a classifier on, however I can't imagine how I ...
3
votes
0answers
33 views

Bag of words: Prediction on new (out-of-sample) data

I'm working with a bag of words in R: library(tm) corpus = VCorpus(textsource) dtm = DocumentTermMatrix(corpus) dtm = as.matrix(dtm) I use the matrix ...
0
votes
1answer
26 views

Target in a dataset for text classification

I have a dataset that lacks the target variable. The dataset contains a list of emails and I would need to classify them as spam/not spam. Do you know how I can build a model that can help me to ...
0
votes
2answers
29 views

k-means and LDA for text classification: how to test accuracy?

I have many tweets that I would like classify based on their similarity. Unfortunately I am not quite familiar with text-classification and nlp, so I had to read a lot of documents before having an ...
2
votes
2answers
38 views

Text Classification : Classifying N classes vs rest of the classes

Apologies if this is naive, I am fairly new to the domain. I have a requirement where I am trying to classify 2 types of text data, i.e, I have got 2 classes to classify my data upon. I am able to get ...
1
vote
2answers
42 views

How many layers should I replace in transfer learning CNN

I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
2
votes
1answer
40 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
2
votes
1answer
27 views

Classify text as logical/ not logical

Can some one advise me direction where to look in.Or some resources. Here is a task: User leaves feed back-text with min 50 characters. I need to check if it's normal human sentences/ word ...
0
votes
2answers
16 views

Need for Dense layer in Text Classifcation

While creating a model for text classification, what is the need for a Dense Layer? I noticed in multiple examples the following is the structure. A softmax is what required right instead of the Dense ...
0
votes
1answer
20 views

Text Classification on a very small data set with a lot of classes

I have a data set consisting of 455 rows spread over 21 different classes. The data set is imbalanced as well as you can see below. ...
0
votes
1answer
24 views

How to handle such a large class imbalance in text data?

I am working on a multi class text classifier. The total number of class that are there is 265 and total number of rows is 20,000. The class with largest number of occurrences has 6000 samples and ...
1
vote
1answer
34 views

How to deal with imbalanced text data

I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance: "Tresemme shampoo and conditioner - sulfate-free" = ...