# How to Classify an Image in a Class and a Subclass?

I am a beginner in Machine Learning and I have a situation where an Image needs to be classified into first a super class, then a subclass. For Example, I have a set of images of fruits, containing images of Mangoes, Banana, Peaches, Apples etc. Then each fruit can either be fresh or rotten. I have training set containing images of rotten and fresh fruits for each Fruit type. So for a test image I need to classify which fruit it is (Mango, Banana, etc) and then whether it is fresh or rotten. I want to know how to approach this kind of problem

## The simple solution

Have one classifier which can classify all the combinations:

• Fresh banana
• Rotten banana
• Fresh mango
• Rotten mango
• ...

## The good solution

Or have one network with multiple outputs. See the master Thesis of Marvin Teichmann for example. https://arxiv.org/abs/1612.07695

So one path of the network would be:

• Fresh vs Rotten

The other path of the network would be

• Banana vs Mango vs ...

The difference to the simple solution is that you can give a probability to the cases, e.g. for the ground truth "fresh banana" you might say 99% banana but only 42% fresh.

## The obvious solution

This one should not be done: have multiple classifiers

• can you explain why the obvious solution is bad? Apr 2 '18 at 9:53
• The results are not so good because there are shared features for both tasks. If you give the network more labeled data it can generalize better. I think in Marvin Teichmann thesis you can read about that, IIRC Apr 2 '18 at 11:32

I want to know how to approach this kind of problem

Yes this is definately possible... (Probably Known as Classification + Localization)

Split the conv-nets into 2 parallel arch's at the very end , before We generally add a single-dense-layer after Feature Extraction, here you will have a split into two...(the image attached will make it clear)

• One to detect the Class i.e Which Fruit it is...
• Second to classify them as Rotten/Fresh

Have a look at this Image(From CS-231N Some credits apply)(in your case, replace Animals class and species with Fruits Class and State)(you don't need a bbox, so a modification to the arch will yield the results as desired provided that the loss is calculated and used correctly on both the ends of the network)

This Paper link probably address the same