Questions tagged [text-classification]
The text-classification tag has no usage guidance.
99
questions
-1
votes
0answers
17 views
Balance in Training set for Text classification [closed]
I am trying right now to find out how much posts on stack overflow are security related with NLP. For this I have a set of 5000 labeled secure related Posts. I have a set of 50k non security related ...
2
votes
0answers
19 views
Does Keras allow using independant classifier
I have spam data set classified into 0:ham or 1:spam. I created the Embedding layer with Keras and then I used Conv1D following with other layers. My question is about adding a classifier like Random ...
1
vote
1answer
13 views
Grouping profiles strings having the same words, but occurring out of order
I have a dataframe containing a column of profile types, which looks like this:
...
0
votes
0answers
16 views
Correlations between target variables and other variables (binary)
I would like to calculate the correlation between the presence of uppercase, special punctuation, specific words, in texts labeled as fake/not fake.
In order to do it, I created some columns in my ...
-1
votes
0answers
11 views
Bechmark models for Text Classification / Sentiment Classification
I am currently working on a novel application in NLP where I try to classify empathic and non-empathic texts. I would like to compare the performance of my model to some benchmark models. As I am ...
3
votes
1answer
27 views
Predictive output with your own model built
I would need to better understand how can be created a machine learning algorithm from scratch using an own model developed based on boolean values, for example # of words in a text, # of punctuation, ...
0
votes
0answers
23 views
How can I make a better unsupervised text classifier model? Is POS tagging part of Machine Learning and Data science?
I have got complaints data, which is not good, and often contains less than 3 words in every complaint (sometimes so short that only one word of them makes sense). The objective is to find what's ...
2
votes
1answer
61 views
Is there a way to classify an alphanumeric string? [closed]
I have a long table containing various items. Each item has a unique code associated with it. As you can see from the sample below, the codes have a pattern to them. What I'd like to do is to create a ...
0
votes
3answers
38 views
Distinguish randomly generated texts from reasonable for human texts [closed]
I have strings short texts of 2 types:
'23jd2032n0d2mn', 'fn830n30rn83', 'fhui29n4ok', 'qn4foml', ...
and
...
0
votes
1answer
33 views
How can a ML algorithm learn to classify fake news? [duplicate]
I am new in Machine learning techniques and in fake news detection by using these algorithms (SVM, nn, logistic regression,..).
I would like to understand how an algorithm can learn from a training ...
0
votes
0answers
13 views
Classify Spanish Text into different Categories
I want to recommend articles to users depending upon what type of article is user reading, Music, Movies, Politics, etc.
I have 3 features: Page Title, Labels, article content.
I am using an API (...
0
votes
1answer
30 views
Ordering of standardization, pca, and/or tfidf for neural network
I have 60k rows of text data. I have tokenized it into 55k columns. I am using a neural network to classify the data but have some questions about how to order my preprocessing steps. I have too much ...
0
votes
0answers
12 views
Trying to use CBOW for tweet classification
I'm trying to use the Continuous Bag Of Words method for word embedding on a corpus of 7503 tweets.
In particular, I'm trying to use CBOW on this Kaggle competition, which involves classifying tweets ...
4
votes
1answer
85 views
How to include categorical fields to enhance a text classification
I would have a question on how to add more categorical fields in a classification problem.
My dataset had initially 4 fields:
...
0
votes
1answer
21 views
Is it possible to classify documents of corpus using labels?
I have a corpus of 23000 documents that need to be classified into 5 different categories. I do not have any labeled data available to me, just freeform text documents and labels(yes, one-word labels, ...
0
votes
1answer
21 views
How to categorize unlabelled promotional email data
I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
2
votes
1answer
20 views
Interpreting confidence interval results for datasets
I have created a dataset automatically and wanted to clarify my interpretation of the amount of noise using the confidence interval.
I selected a random sample and manually annotated the sample and ...
0
votes
1answer
38 views
Classifiers and accuracy
I would like to ask you how to use classifier and determine accuracy of models.
I have my dataset and I already cleaned the text (remove stopwords, punctuation, removed empty rows,...).
Then I split ...
0
votes
0answers
17 views
How to apply an algorithm for text classification?
I would like to know more about algorithms to apply for text classifications.
I know that steps include also remove of punctuation and stop words and tokenisation to clean the text. If I already ...
0
votes
1answer
19 views
How to create a Document Categorization Classifier for different contexts of Documents
I have a doubt solving a test. The idea here is to demonstrate the NLP and Machine Translation abilities.
The Dataset is a multilingual, multi-context set of documents. The dataset is divided on ...
1
vote
2answers
28 views
Problem of continuous training - Supervised learning
I am sure this is a most common problem, but would like to know by experts on how to tackle it. Note that, I mostly deal with textual data (NLP problems).
When a supervised learning model is created, ...
1
vote
1answer
39 views
Creating a valid dataset for obtaining results
I have created a domain-specific dataset, lets say it is relating to python programming topic posts. I have taken data from various places specific to this topic to create positive examples in my ...
0
votes
0answers
15 views
How to approach a text parsing problem
I'm looking to parse human-generated cyber-physical sensor tags into a series of standardised tags and identifiers.
For example "Rm3.ZnT-SP" refers to ROOM ID:3 ZONE TEMPERATURE SETPOINT
The ...
2
votes
1answer
39 views
Any useful tips on transfer learning for a text classification task
I am doing a supervised binary text classification task.
I want to classify the texts from site A, site B, and site C.
The in-domain performance looks OK for texts of each site. (92%-94% accuracy).
...
0
votes
1answer
20 views
(Serious) Dataset of paedophilic Youtube comments (or similar)? [closed]
I think it would be useful to create a model that tries to predict whether a youtube comment is paedophilic - maybe the model should also take into account the channel name/description/front image.
It'...
1
vote
1answer
66 views
Text classification with Word2Vec on a larger corpus
I am working on a small project and I would like to use the word2vec technique as a text representation method. I need to classify patents but I have only a few of them labelled and to increase the ...
1
vote
1answer
27 views
Identifying templates from SMS text
I am building an app where I identify information from the SMS, something similar to expense management apps. I have a parser which reads all the SMS of user, identifies SMS of interest and parses ...
0
votes
1answer
42 views
Sklearn SVM question classification
So, I have found that there are many ways to classify words with sklearn's SVM algorithm. But I want to classify questions by taxonomy, as shown in the following dataset:
The goal of this task is to ...
0
votes
0answers
53 views
Hierarchical transformer for document classification: error with model implementation, and advice on extracting attention weights
I am trying to implement a hierarchical transformer for document classification in Keras/tensorflow, in which:
(1) a word-level transformer produces a representation of each sentence, and attention ...
2
votes
1answer
52 views
Is there any way to plot ROC curve for Ensemble hard voting classifier?
I am working on a multi-class text classification problem and performing an Ensemble learning for text classification. I chose hard voting as ensemble technique. I tried to plot ROC curve for my ...
0
votes
0answers
19 views
Supervised Text Classification
I need to create a summarized report showing how many containers are being imported by each company. This report needs to be generated on monthly basis. The company names are entered in a way that ...
2
votes
2answers
26 views
How to identify text similarity based on training data?
I have a set of documents (1 to 11) for which the labeling is done.
Lets Assume:
...
-1
votes
1answer
114 views
0
votes
0answers
24 views
Algorithm for rectifying incorrect text from OCR
I have an OCR (tesseract) engine running on processing "chyron" text from a live stream news feed.
However, the OCR is often inaccurate and ends up with some noisy words:
...
0
votes
0answers
29 views
use genetic algorithm as a feature selection for text classification
how to apply the genetic algorithm as a feature selection for text classification in python
I need to use GA to select most relevant feature in text classification
3
votes
0answers
37 views
Bag of words: Prediction on new (out-of-sample) data
I'm working with a bag of words in R:
library(tm)
corpus = VCorpus(textsource)
dtm = DocumentTermMatrix(corpus)
dtm = as.matrix(dtm)
I use the matrix ...
2
votes
2answers
39 views
Text Classification : Classifying N classes vs rest of the classes
Apologies if this is naive, I am fairly new to the domain.
I have a requirement where I am trying to classify 2 types of text data, i.e, I have got 2 classes to classify my data upon. I am able to get ...
0
votes
1answer
28 views
K-means and LDA for text classification
I hope to explain in a clear way what I would like to do. I have more than 50000 tweets and I would like to add some labels on topics. So I have used LDA for doing this. I have also used k-means to ...
0
votes
2answers
29 views
k-means and LDA for text classification: how to test accuracy?
I have many tweets that I would like classify based on their similarity.
Unfortunately I am not quite familiar with text-classification and nlp, so I had to read a lot of documents before having an ...
1
vote
2answers
64 views
How many layers should I replace in transfer learning CNN
I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
1
vote
2answers
26 views
Confusion with using different classes in neural networks (training vs testing)
I am new to deep learning and I am confused about having a neural network trained on certain classes and tested on different ones. Suppose I want to have a convolutional neural network that learns ...
2
votes
1answer
27 views
Classify text as logical/ not logical
Can some one advise me direction where to look in.Or some resources. Here is a task:
User leaves feed back-text with min 50 characters.
I need to check if it's normal human sentences/ word ...
0
votes
2answers
16 views
Need for Dense layer in Text Classifcation
While creating a model for text classification, what is the need for a Dense Layer? I noticed in multiple examples the following is the structure. A softmax is what required right instead of the Dense ...
0
votes
1answer
22 views
Text Classification on a very small data set with a lot of classes
I have a data set consisting of 455 rows spread over 21 different classes. The data set is imbalanced as well as you can see below.
...
0
votes
4answers
57 views
Clusters: how to improve results for text classification
I am trying to classify texts using kmeans, TfidfVectorizer, PCA. However, it seems that many texts are not correctly classified as you can see: I have texts in cluster2 that should be in Cluster 0 or ...
0
votes
0answers
29 views
How to properly vectorise when I have several text features?
I am having several features as text and I'm using them once for a classification problem and then later for a regression problem.
The textual data features themselves aren't categorical. Because as ...
1
vote
1answer
33 views
Which insights a data scientist could derive from text-analysis? [closed]
I have many texts and I am trying to analyse them. After tokenising them, studying words frequency, spotting any typos, studying punctuations, I have been working on POS tagging.
Since it is my first ...
0
votes
0answers
25 views
Text Classification One-shot learning (1sample/class ~1000 classes)
I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000).
...
2
votes
1answer
27 views
How to use ontologies for text classification?
I am new to machine learning and want to classify sentences using ontologies (taxonomies/ knowledge graphs) and supervised learning methods (I have an annotated training dataset).
My question is how ...
0
votes
0answers
9 views
Dirichlet smoothing as an IDF component
How can Dirichlet smoothing be used as an IDF component to estimate the probabilities of a Topic model ?
i.e, Smoothing with a background collection model to estimate topic model ?
I've seen many ...
