# Compare two topic modelling sets

I have two sets of newspaper articles where I train the first newspaper dataset separately to get the topics per each newspaper article.

E.g., first newspaper dataset
article_1 = {'politics': 0.1, 'nature': 0.8, ..., 'sports':0, 'wild-life':1}


Again, I train my second newspaper dataset (from a different distributor) to get the topics per each newspaper article.

E.g., second newspaper dataset (from a different distributor)
article_2 = {'people': 0.3, 'animals': 0.7, ...., 'business':0.7, 'sports':0.2}


As shown in the examples, the topics I get from the two datasets are different, thus I manually matched similar topics based on their frequent words.

I want to identify whether the two newspaper distributors publish the same news in every week.

Hence, I am interested in knowing if there is a systematic way of comparing the topics across two corpora and measuring their similarity. Please help me.

• Interesting question. what is the technique you used for topic-modeling? – Volka Oct 13 '17 at 1:07
• I would find a pooled topic model, then compare the individual distributions (e.g., by KLD). – Emre Oct 16 '17 at 1:07
• I mean find the topics assuming all the articles come from a common source. KLD is en.wikipedia.org/wiki/Kullback–Leibler_divergence – Emre Oct 16 '17 at 1:44
• I would use a hierchical model that defines the newspaper Dirichlet distributions based on the pooled topic model. You should be able to implement this in Edward; see this GitHub discussion. Once you have the newspaper distributions, the difference can be defined by the KLD or the symmetric Jensen-Shannon divergence as follows: bariskurt.com/… Good luck! – Emre Oct 17 '17 at 4:30
• This lecture might help. Read about LDA, Dirichlet distribution, and divergence measures. – Emre Oct 17 '17 at 23:59

Considering you extracted the news using a TF-IDF approach what you have as a result is just one feature (frequency of terms). I think you would need to add more features to your corpus to be able to match two news as the same (or similar).

One new feature would be the temporal, where you would add a timestamp for the news. It will allow you check if two news (from different publishers) were published on the same period. (a week, 2 weeks, etc).

The second one could be a spatial, if you have geolocation of the news for example, you can add it to your train dataset. Something similar was done by Chi-Chun Pan. It would allow you be more confident if the news happened on the same place.

One method to compare the topics across two corpora and measuring their similarity is with Kullback-Leibler divergence, aka relative entropy. Kullback-Leibler divergence is a measure of how one probability distribution diverges from a second probability distribution.

Another, more scalable algorithm can be found in Topic Model Diagnostics: Assessing Domain Relevance via Topical Alignment

• Thanks a lot for your great answer. Can you please elaborate how to use your second option (vis.stanford.edu/topic-diagnostics) as it is not very clear to me (as how to apply it to my problem) – Smith Oct 21 '17 at 13:28