Suppose that the sample set consists of labelled data, where each label corresponds to a class (say a sub-topic), and every class belongs to a group (a topic). The model should be able to predict the sub-topic given a text, and there is no general interest in predicting the topic directly. So far I have tried a bi-directional LSTM model using the "top" sub-topics (i.e. those that have the most data associated to them) as allowing many sub-topics reduces the accuracy quite dramatically.
In order to improve on this model, I would like to come up with a topology that allows me to first classify by topic and then use this information (together perhaps with the initial data, fed back into the model at later layers), to predict the sub-topic.
A naive approach would be to build a model that deals with the topics and use its output as the input for models trained on the sub-topics for each topic. However, I believe that a single DNN model would perform better than this simplistic approach.
What DNN topology would allow me to achieve my goal?