I am putting together a multi-category classification algorithm. Since it's NLP, the training data is very simple with one column for labels and another column for text. However, because it's NLP, some training records can fit multiple categories. Should I iterate through & have multiple entries (same data, different labels) or should each piece of data only have 1 category assigned?
$\begingroup$
$\endgroup$
1
asked Mar 5, 2018 at 19:45
-
$\begingroup$ Your question is not entirely clear to me. As far as I understand the training data you have is a list of pairs of text and a single categorical values. Correct? Is that data a given or are you constructing the data yourself? $\endgroup$– S van BalenMar 8, 2018 at 7:48
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
No, it is perfectly possible to train on multiple categories. What you need, though, is an exhaustive list of these categories (in supervised learning, that is).
Suppose you are trying to associate sentences with topics, and you have a list of possible topics topics = ['sports', 'soccer', 'politics'].
It sounds like your data look something like this:
sentence | topics
-------------------------------|----------------------------------
'Barack Obama loves soccer' | ['politics', 'sports', 'soccer']
'The parliament is important' | ['politics']
'Soccer is fun' | ['sports', 'soccer']
Then you need to one-hot encode the topics:
X = [['Barack Obama loves soccer'], ['The parliament is important'], ['Soccer is fun']]
Y = [[1, 1, 1], [1, 0, 0], [0, 1, 1]]
And then you train a neural network to predict not one but three (= number of topics) values.
-
$\begingroup$ Yes, that's definitely similar to the data I have. But does it have to be hot-encoded in the manner you listed? Is there a danger in having multiple entries? Same text, but different (single) topic per entry? My desired prediction is the one topic that it fits best. $\endgroup$ Mar 8, 2018 at 16:10
-
$\begingroup$ Neural networks only accept numerical in- and outputs, so you will have to come up with some encoding anyways. Could you be more specific about your training set? If a sentence has multiple topics, how should the network know which one fits 'best'? $\endgroup$ Mar 8, 2018 at 16:19
-
$\begingroup$ Hmmm . . . let me review a classifier with you. (1) the model you proposed isn't strictly "one hot" encoding and (2) most classifiers return an array of topic probabilities that you can rank on your own and choose. Some of those probabilities will be 0 if it doesn't match at all and others will be a high number because the classifier thinks it fits it best. $\endgroup$ Mar 8, 2018 at 16:23
-
1$\begingroup$ You are right, 'one-hot' is not the right word for this. Maybe 'dummy variables' fits better? About the probabilities: Yes; but be careful with your interpretation. If a certain topic receives a number close to 1 that means your classifier is very certain that the sentence belongs to that topic. That does not necessarily mean that this topic is the single best description of the sentence. Your classifier could be very certain because it has seen many similar training examples. $\endgroup$ Mar 8, 2018 at 20:57
-
$\begingroup$ sklearn calls this label binarizaing, but I agree that "dummifying" or some variant works. I wouldn't call it "dummy variables" though: without a descriptive like target/dependent/endogenous, "variables" usually refers to the input features. $\endgroup$ Mar 14, 2018 at 6:32