5
$\begingroup$

If a training set has 5 classes including : "label-A", "label-B", "label-C", "label-D", "others".

But the problem is much simpler, it wants to distinguish each input whether it is belongs to "label-ABCD" or "others". In this case, there are two solutions to solve this problem in my mind.

Solution-1 : Train a 5-classes classifier, when the classifier predicts the input as "label-A" or "label-B" or "label-C" or "label-D", we relabel it as "label-ABCD".

Solution-2 : Train a 2-classes classifier, we relabel the data as "label-ABCD" which is labeled as "label-A" or "label-B" or "label-C" or "label-D". And then it becomes a binary-classification problem.

My question is...

  1. Which way can the model get a better performance in "theorem"?

  2. In real case, this two cases get almost the same performance by CNN classification model, and I am wondering if I adopt a weaker classifier like C4.5, Naive Bayes, SVM...which way will win?

Thanks!!

$\endgroup$
4
$\begingroup$

The greater the number of output nodes the higher complexity you will add to your model. This means that given a fixed amount of data, a greater number of output nodes will lead to poorer results. I would use a ABCD vs. others strategy.

Instead of conditioning your model to learn the distributions of the class A, B, C and D separately you will combine them. This means that is A and B are different in some way, but this difference is irrespective of the classification with "others" then there is no need to learn that distinction.

For example: if you want to detect dog, cat, human with features such as weight, height and number of legs. The number of legs feature will have relatively low importance, because cats and dogs will likely all have 4 legs. However if I want to classify cat/dog vs humans, then the number of legs will be the most important feature. It might be the only feature you need.

One caveat may be severe class imbalance. By combining your classes in this way you may end up with an over representation of ABCD. You can use techniques such as anomaly detection to train a model on your ABCD data and then detect whether a novel instance falls within this distribution, or is an outlier in which case you would label it as "others."

$\endgroup$
  • $\begingroup$ In my opinion, if there is a zoo only contains tigers and lions, and given a picture of animals, we want to predict whether the animals is belongs to the zoo. If the input is liger(has lion mom & tiger dad), it has both lion and tiger's features so the model may confuse and predict it as one of the zoo's member, however, if the model has ability to identify lion and tiger individually, then the model may know two things, first liger is not a lion, second liger is not a tiger. Finally we know that liger is not belongs to the zoo, because it is neither a lion nor a tiger. $\endgroup$ – PoCheng.Lin Feb 9 '18 at 10:10
  • $\begingroup$ So my conclusion is, multiclass model may understand more detailed of each class, so it can figure out the problem above. $\endgroup$ – PoCheng.Lin Feb 9 '18 at 10:10
  • $\begingroup$ If we put all the class together (ABCD), does it extended a problem that the model can only extract the ABCD's feature roughly. Once we have the instance of A-. B+, C+ etc..., then The model may predict them as label-ABCD $\endgroup$ – PoCheng.Lin Feb 9 '18 at 10:10
  • $\begingroup$ Considering your first comment. Should the liger fall within the zoo which is comprised entirely of tigers and lions? Moreover, consider the strength of machine learning, I don't think even a human would be able to perform this task reliably. But yes your conclusion may understand more details about each class, but in doing so it learns difference between classes that it might not otherwise need to consider if you would be to group the classes. Such as my dog/cat vs. others example. $\endgroup$ – JahKnows Feb 9 '18 at 11:01
  • 1
    $\begingroup$ Are examples from a similar distribution of A-, B+, C+ represented in the training set labelled as "others". If not, this is just a wildcard, it's very hard to predict. Don't forget the training data should be collected from the same distribution as the data with which you will apply your model. $\endgroup$ – JahKnows Feb 9 '18 at 11:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.