In short, the question is: how can I build a regularly updated chain of topics which would also show how topics emerge and disappear over time?
To be more precise:
- I have a data with timestamps updated with a certain regularity - say, each week I get another batch of forum posts.
- I want to be able to trace topics in a sequential manner - so topic weights change over time, some disappear as well as new ones appear.
It is not hard to trace changes in topic weights over time (train and run the model, assigning document-topic weights to each document and then plot with timestamps). But the second part part (topic disappearance and emergence) is problematic. All I have found/come up with:
- Online LDA can supposedly do the job, and there is a scikit-learn implementation, but I can't find any practical examples. I'm not sure I will be able to tune a model well without them, as my understanding of the underlying math is superficial.
- There's been works on other models that could help (e.g. here, here), but it seems there are no implementations in Python or R. could you suggest an approach that could help?