So I have a CNN task at hand. Let's say I have a dataset containing pictures of a number of classes, their attributes and sub attributes and so on.

What is the state of art/How can we design a system which first predicts the class, then based on that class the attributes which are specific to that class e.g claws present in birds but not in humans and so on.

How is such a system designed with minimum resource utilization?

Note: My question is not about prediction of classes and attributes but it is the design of a system which will predict the class then go predict the specific attributes belonging to the class and so on.

  • $\begingroup$ You could use a common set of attributes, then enforce relationships in your loss function (setting a high cost for disallowed combinations). I believe using a hierarchy should improve efficiency. Welcome to the site! $\endgroup$
    – Emre
    Aug 2 '18 at 1:30
  • $\begingroup$ @Emre hmmm... I'll try it theoretically but from first view it doesn't look the most efficient...But let's see where it stands among other answers $\endgroup$
    – DuttaA
    Aug 2 '18 at 1:57
  • $\begingroup$ From what I understood you might be interested in Hierarchical Classification where top classes share coarse set of features and in a hierarchical way the subclasses present other set of features on top of those high-levels to obtain better classification, e.g. have you seen this: arxiv.org/abs/1709.09890, or link.springer.com/article/10.1007/s11042-017-5443-x? Or maybe this somewhat relevant arxiv.org/abs/1410.0736 (with Keras implementation: github.com/justinessert/hierarchical-deep-cnn)? $\endgroup$ Aug 2 '18 at 6:31

I'm not sure what the state of the art is right now, but I had a similar problem for which I designed a neural network. What I did was make a model that predicted the class probabilities from the inputs (in your case the image) and then made another model that took both the normal inputs and my probability predictions. I saw a very large increase in the accuracy for the second part after I had fed the class predictions into the feature prediction section.

You could design this as one model with two outputs where the class is an intermediate layer but still an output. Your loss function would be adding the loss for the class prediction and the feature prediction. Weighting the loss for the class prediction would cause the model to lean more towards correctly predicting the class over the other features. If you must absolutely predict the class first and then the features, you can first train the model on the class loss, then train it again with the feature loss after freezing the weights associated with/preceding the class predictions.

  • 1
    $\begingroup$ Thanks for the answer..This is the most general approach which I have been thinking..Let's see if someone comes up with something clever $\endgroup$
    – DuttaA
    Aug 2 '18 at 1:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.