Equally sized topics in Latent Dirichlet allocation

Question

I'm using the topicmodels package for R to cluster a big set of short texts (between 10-75 words) into topics. After manually reviewing a few models it seems like there are 20 realtivly stable topics. However, what I find really weird is that they are all roughly the same size! Each topic catches around 5% of tokens and 5% of texts. In terms of the tokens, the smallest topic is 4.5% the largest 5.5%.

Can anybody suggest if this a 'normal' behaviour? This is the code I'm using:

ldafitted <- LDA(sentences.tm, k = K, method = "Gibbs",
             control = list(alpha = 0.1, # default is 50/k which would be 2.5.  a lower alpha value places more weight on having each document composed of only a few dominant topics
                            delta = 0.1, # default 0.1 is suggested in Griffiths and Steyvers (2004).
                            estimate.beta = TRUE,
                            verbose = 50, # print every 50th draw to screen
                            seed = 5926696,
                            save = 0,    # can save model every xth iteration
                            iter = 5000, 
                            burnin = 500,
                            thin = 5000, #  every thin iteration is returned for iter iterations. Standard is same as iter
                            best = TRUE)) #only the best draw is returned

In short: My question is if there are circumstances under which it is reasonable that Latent Dirichlet allocation will cluster text in topics of equal size? Or is it something I should be worried if it happens?

Answer 1

It is normal, the best explantation I found for it, is from physics. Since Gibbs Sampling was known in physics long before LDA and LDA can simply be seen as a kind of matrix factorization. There is a system which has particles (words) and the particles can be in different states (topics). States with lower energy have a higher probability of being occupied than states with higher energy. Or simply: Since LDA is a dimensity reduction method that just compresses a words x documents matrix into a topics x documents and a words x topics matrix, the most effective way to do it is by maximizing the entropy which is done if the clusters get equal size.

Just noticed that you can simply derive the above assumption from the dirichlet distribution itself: Take a look at the density (PDF) function and ignore the normalizing factor at the front and only assume symmetric alpha parameter. This results in a product term of X1...XK, that product term gets it's maximum if all X1...XK's have equal size, since the sum X1+X2+...+XK = 1, it's the same case as entropy above.