Skip to main content

Questions tagged [text-classification]

For questions about text classification, the task of assigning predefined categories (or classes) to free-text documents.

Filter by
Sorted by
Tagged with
5 votes
1 answer
941 views

How to preprocess with NLP a big dataset for text classification

TL;DR I've never done nlp before and I feel like I'm not doing it in the good way. I'd like to know if I'm really doing things in a bad way since the beginning or ...
gabriel garcia's user avatar
1 vote
1 answer
140 views

what kind of algorithm should I use to classify the text data example given?

What kind of classification or learning algorithm that suits this kind of data example If I have to build a model using the given key words then predict column B and then to column A? what kind of ...
User123456's user avatar
1 vote
1 answer
148 views

Grouping profiles strings having the same words, but occurring out of order

I have a dataframe containing a column of profile types, which looks like this: ...
Erin's user avatar
  • 81
71 votes
4 answers
95k views

What is purpose of the [CLS] token and why is its encoding output important?

I am reading this article on how to use BERT by Jay Alammar and I understand things up until: For sentence classification, we’re only only interested in BERT’s output for the [CLS] token, so we ...
user3768495's user avatar
2 votes
1 answer
790 views

Should bag of words in training set include test set data when doing text classification?

I'm doing text classification to identify 'attacks' from Wikipedia comments using a simple bag of words model and a linear SVM classifier. Because of class imbalance, I'm using the F1 score as my ...
jonnyf's user avatar
  • 121
1 vote
2 answers
1k views

What effect repetitive data will have on the performance of the model

I understand that my question is very broad and that the correct answer may depend on various things. I want to get an idea in general what we may expect if we have repetitive data in our dataset. ...
Maria's user avatar
  • 331
1 vote
0 answers
61 views

How do I use a machine learning or AI algorithm on this kind of data?

I have a text data set with keyword, thematic, Sub thematic, intent Now I have to train a keyword then first classify into sub thematic, then into thematic and then into intent. the data looks like ...
j123's user avatar
  • 11
1 vote
0 answers
1k views

Can SMOTE be applied over sequence of words (sentences)?

I have a highly unbalanced text classification data. I am trying to over-sample through SMOTE. I have a doubt that applying SMOTE over sequence of word indices will give me valid data points or not (...
SHASHANK GUPTA's user avatar
0 votes
1 answer
3k views

recognizing the correct word & "Set type is unordered"-error in python-pandas

My Data Set (CSV): ...
Jsmoka's user avatar
  • 13