I'm training an LDA model with gensim's LdaMulticore. The topics look great, but knowing the domain I know there exists topics within topics but I'm not quite sure the best way to model this.
I'm currently doing this:
1) train LDA model on all records to get general topics
2) use this LDA model to assign each record a primary topic
3) for each topic, retrieve only the records that were assigned that topic
4) train a new LDA model only on the filtered records (for example, where topic ID == 3) to generate sub topics from filtered set
5) Assign each record a general topic ID and a sub topic ID
Is this a valid way to get topics and sub topics? Should I be weary of this approach?
Thanks for any insight.