1
$\begingroup$

In short, the question is: how can I build a regularly updated chain of topics which would also show how topics emerge and disappear over time?

To be more precise:

  • I have a data with timestamps updated with a certain regularity - say, each week I get another batch of forum posts.
  • I want to be able to trace topics in a sequential manner - so topic weights change over time, some disappear as well as new ones appear.

It is not hard to trace changes in topic weights over time (train and run the model, assigning document-topic weights to each document and then plot with timestamps). But the second part part (topic disappearance and emergence) is problematic. All I have found/come up with:

  • Online LDA can supposedly do the job, and there is a scikit-learn implementation, but I can't find any practical examples. I'm not sure I will be able to tune a model well without them, as my understanding of the underlying math is superficial.
  • There's been works on other models that could help (e.g. here, here), but it seems there are no implementations in Python or R. could you suggest an approach that could help?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

The state of the art method for dynamic/diachronic topic modelling is Dynamic Embedded Topic Models. The author published an implementation in python.

I think that the previous state of the art approach was dynamic LDA. There were several variants of this, probably online LDA is one of them.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the suggestion, but these two do not address the main issue: tracing emerging of new topics over time. $\endgroup$
    – yys
    Oct 3, 2021 at 17:42
  • $\begingroup$ @yyz the emergence of a new topic can be obtained from the probability of topic given time that these two models estimate: for every topic $z$ you can obtain the evolution of the topic across time $p(z|t)$. You might need to define a formal condition about what is "emergence" for your data, it depends how accurately the model finds them. $\endgroup$
    – Erwan
    Oct 3, 2021 at 19:17
  • $\begingroup$ Could you please clarify a little? Suppose I have two time slices, and my assumption is that the composition of the topic set changes in the second one (i.e. some new topics have appeared). I train a model on both slices, setting an N of topics to, say, 20. Then I fit to both slices separately, and, in case my assumption is true, I should notice higher weights for some topics in the second slice (or these weights should surpass some deliberate threshold which I think implies "emergence"). Is this about right? $\endgroup$
    – yys
    Oct 4, 2021 at 6:52
  • $\begingroup$ @yyz the method you're describing makes perfect sense, but this is not really what is called diachronic/dynamic in the literature: a dynamic model like DETM or DLDA is meant to represent time as a dimension inside the model, and it makes an assumption that topics evolve continuously across time. These dynamic models are intended to be used with more than 2 time slices, for example if you have a dataset with time-stamped documents spanning for instance 50 years.... $\endgroup$
    – Erwan
    Oct 4, 2021 at 19:01
  • $\begingroup$ ... The method you describe is based on using a regular "static" model (e.g. LDA or Embedded Topic Models) on the 2 time slices: by fitting a model for t1 and t2 jointy and then independently on each of them you can indeed match the topics between t1 and t2, and capture the ones for which the proportion changes significantly. If your data contains only 2 times slices I think this is the right method. If it contains a larger span of discrete time units then a dynamic model should give better results. $\endgroup$
    – Erwan
    Oct 4, 2021 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.