Questions tagged [text-classification]
The text-classification tag has no usage guidance.
79
questions
1
vote
1answer
32 views
Creating a valid dataset for obtaining results
I have created a domain-specific dataset, lets say it is relating to python programming topic posts. I have taken data from various places specific to this topic to create positive examples in my ...
0
votes
0answers
13 views
How to approach a text parsing problem
I'm looking to parse human-generated cyber-physical sensor tags into a series of standardised tags and identifiers.
For example "Rm3.ZnT-SP" refers to ROOM ID:3 ZONE TEMPERATURE SETPOINT
The ...
2
votes
1answer
33 views
Any useful tips on transfer learning for a text classification task
I am doing a supervised binary text classification task.
I want to classify the texts from site A, site B, and site C.
The in-domain performance looks OK for texts of each site. (92%-94% accuracy).
...
0
votes
1answer
19 views
(Serious) Dataset of paedophilic Youtube comments (or similar)? [closed]
I think it would be useful to create a model that tries to predict whether a youtube comment is paedophilic - maybe the model should also take into account the channel name/description/front image.
It'...
1
vote
1answer
23 views
Text classification with Word2Vec on a larger corpus
I am working on a small project and I would like to use the word2vec technique as a text representation method. I need to classify patents but I have only a few of them labelled and to increase the ...
1
vote
1answer
25 views
Identifying templates from SMS text
I am building an app where I identify information from the SMS, something similar to expense management apps. I have a parser which reads all the SMS of user, identifies SMS of interest and parses ...
0
votes
1answer
32 views
Sklearn SVM question classification
So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset:
The goal of this task is to ...
0
votes
0answers
26 views
Hierarchical transformer for document classification: error with model implementation, and advice on extracting attention weights
I am trying to implement a hierarchical transformer for document classification in Keras/tensorflow, in which:
(1) a word-level transformer produces a representation of each sentence, and attention ...
-1
votes
0answers
42 views
I need help on a text classification problem of dividing the postal addresses into valid or invalid
The problem is a text classification problem of dividing the addresses into valid or invalid. A valid address is the one where parcels can be delivered(postally correct address)
The csv files have the ...
2
votes
1answer
23 views
Is there any way to plot ROC curve for Ensemble hard voting classifier?
I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
0
votes
0answers
16 views
Supervised Text Classification
I need to create a summarized report showing how many containers are being imported by each company. This report needs to be generated on monthly basis. The company names are entered in a way that ...
2
votes
2answers
21 views
How to identify text similarity based on training data?
I have a set of documents (1 to 11) for which the labeling is done.
Lets Assume:
...
-1
votes
1answer
23 views
0
votes
0answers
22 views
Algorithm for rectifying incorrect text from OCR
I have an OCR (tesseract) engine running on processing "chyron" text from a live stream news feed.
However, the OCR is often inaccurate and ends up with some noisy words:
...
0
votes
0answers
22 views
use genetic algorithm as a feature selection for text classification
how to apply the genetic algorithm as a feature selection for text classification in python
I need to use GA to select most relevant feature in text classification
3
votes
0answers
30 views
Bag of words: Prediction on new (out-of-sample) data
I'm working with a bag of words in R:
library(tm)
corpus = VCorpus(textsource)
dtm = DocumentTermMatrix(corpus)
dtm = as.matrix(dtm)
I use the matrix ...
1
vote
2answers
35 views
Text Classification : Classifying N classes vs rest of the classes
Apologies if this is naive, I am fairly new to the domain.
I have a requirement where I am trying to classify 2 types of text data, i.e, I have got 2 classes to classify my data upon. I am able to get ...
0
votes
1answer
19 views
K-means and LDA for text classification
I hope to explain in a clear way what I would like to do. I have more than 50000 tweets and I would like to add some labels on topics. So I have used LDA for doing this. I have also used k-means to ...
0
votes
2answers
25 views
k-means and LDA for text classification: how to test accuracy?
I have many tweets that I would like classify based on their similarity.
Unfortunately I am not quite familiar with text-classification and nlp, so I had to read a lot of documents before having an ...
1
vote
2answers
38 views
How many layers should I replace in transfer learning CNN
I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
1
vote
2answers
24 views
Confusion with using different classes in neural networks (training vs testing)
I am new to deep learning and I am confused about having a neural network trained on certain classes and tested on different ones. Suppose I want to have a convolutional neural network that learns ...
2
votes
1answer
25 views
Classify text as logical/ not logical
Can some one advise me direction where to look in.Or some resources. Here is a task:
User leaves feed back-text with min 50 characters.
I need to check if it's normal human sentences/ word ...
0
votes
2answers
14 views
Need for Dense layer in Text Classifcation
While creating a model for text classification, what is the need for a Dense Layer? I noticed in multiple examples the following is the structure. A softmax is what required right instead of the Dense ...
0
votes
1answer
16 views
Text Classification on a very small data set with a lot of classes
I have a data set consisting of 455 rows spread over 21 different classes. The data set is imbalanced as well as you can see below.
...
1
vote
0answers
21 views
How to group news by topics when they do not have labels
I have a dataset that includes 20000 news titles and 9 columns that are not relevant (boolean values) for labelling them.
I have tried to extract top words using tokenizetion and tf-idf for n-grams. ...
0
votes
4answers
52 views
Clusters: how to improve results for text classification
I am trying to classify texts using kmeans, TfidfVectorizer, PCA. However, it seems that many texts are not correctly classified as you can see: I have texts in cluster2 that should be in Cluster 0 or ...
0
votes
0answers
26 views
How to properly vectorise when I have several text features?
I am having several features as text and I'm using them once for a classification problem and then later for a regression problem.
The textual data features themselves aren't categorical. Because as ...
1
vote
1answer
32 views
Which insights a data scientist could derive from text-analysis? [closed]
I have many texts and I am trying to analyse them. After tokenising them, studying words frequency, spotting any typos, studying punctuations, I have been working on POS tagging.
Since it is my first ...
0
votes
0answers
14 views
Text Classification One-shot learning (1sample/class ~1000 classes)
I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000).
...
2
votes
1answer
19 views
How to use ontologies for text classification?
I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset).
My question is how ...
0
votes
0answers
7 views
Dirichlet smoothing as an IDF component
How can Dirichlet smoothing be used as an IDF component to estimate the probabilities of a Topic model ?
i.e, Smoothing with a background collection model to estimate topic model ?
I've seen many ...
0
votes
1answer
24 views
Target in a dataset for text classification
I have a dataset that lacks the target variable.
The dataset contains a list of emails and I would need to classify them as spam/not spam.
Do you know how I can build a model that can help me to ...
0
votes
0answers
15 views
Calculating accuracy in Extractive Summarization
I am trying to implement a text summarization model. I am using keras tensorflow anad I have used bert sentence embeddings and the output of the embeddings are feeded into an LSTM and then to a Dense ...
0
votes
0answers
26 views
How to label data to build a model from scratch
I have different texts, some of them have been manually labelled some others have not.
Texts
“The modelling dataset were rebuilt yesterday”
“This movie is awful”
“Trump said that ufo exist”
I would ...
1
vote
0answers
21 views
Spam/ham classification
I am exploring the use of lime for spam/ham categorisation.
specifically I have a data frame having list of messages.
I would need to identify which messages are spam and which ones are ham by using a ...
1
vote
1answer
20 views
Text classification: accuracy [closed]
I would like to understand how to compute the accuracy of cluster analysis.
I have hundred of texts. The dataset looks like as follows:
...
0
votes
1answer
15 views
text classification : comparing classification reports
I have a 4-leabelled text classification problem.
Could someone help me choose among the below text classifiers ?
I was advised to select the second one ( the one which uses both unigrams and ...
0
votes
0answers
8 views
How to classify very short text for spend analytics?
I have a very small description of the item, example below-
ACCELATOR SHAFT
ADAPTOR –EGR PIPE & EGR VALVE(125KVA)
ADAPTOR-EGR COOLER AND VALVE-125 KVA
ADJUSTING LATCH - ALTERNATOR-125 KVA
Some ...
2
votes
1answer
33 views
Text classification based on n-grams and similarity
I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
0
votes
1answer
22 views
How to handle such a large class imbalance in text data?
I am working on a multi class text classifier. The total number of class that are there is 265 and total number of rows is 20,000. The class with largest number of occurrences has 6000 samples and ...
0
votes
1answer
15 views
Correlation between words, then texts
I have texts collected on different topics. I would like to study the possible correlation between these.
I have started looking at the word frequency and it seems that in one dataset the word with ...
0
votes
1answer
24 views
Text classification analysis based on similarity
I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
-1
votes
1answer
29 views
How to best handle imbalanced text classification with Keras?
I implemented a text classification model using Keras.
Most of the datasets that I use are imbalanced. Therefore, I would like to use SMOTE to handle said imbalance.
I tried both on plain text, and ...
0
votes
0answers
11 views
Padding texts before input for Neural Network
I am trying to classify long texts (documents) using neural networks. I am looking for tutorials and code examples in Keras and everything I find pads all texts to the size of the longest one in the ...
0
votes
0answers
18 views
How to build a dictionary for sentiment analysis?
I would like to know if it would be possible to build a dictionary using terms/words included in a dataset, in order to be used for sentiment analysis (and build a sentiment analysis classification ...
0
votes
1answer
19 views
Text classification and predictive model
I have collected a lot of data that I would like to analyse and classified. Unfortunately, they are not already labelled, so I am going to do manually.
The dataset consists of texts in Italian and I ...
1
vote
1answer
34 views
How to deal with imbalanced text data
I am working on a problem where I have to classify products into multiple classes (more than one) based on product descriptions. For instance:
"Tresemme shampoo and conditioner - sulfate-free" = ...
1
vote
1answer
21 views
Building a text classification model from scratch
I do not know if this is the right place to post my question.
I am a beginner in data science and machine learning techniques. I would need to build a modem that can allow me to classify texts and run ...
0
votes
0answers
7 views
Single numeric features row and multiple text rows corresponding to numeric data. How to classify in dataset?
For example, take a review of a product. I have numeric information corresponding to the product and also multiple reviews and I have to output a binary class for the product. How can I predict when I ...
0
votes
1answer
21 views
Text Mapping - Medicine Names
We have a problem where we have a standardized database of Medicine names. On the other hand, there is a subset of medicine names which could have spelling mistakes, different structure or hypens, ...
