4
$\begingroup$

I'm trying to understand the technical part of Latent Dirichlet Allocation (LDA), but I have a few questions on my mind:

First: Why do we need to add alpha and gamma every time we sample the equation below? What if we delete the alpha and gamma from the equation? Would it still be possible to get the result?

LDA sampling formular

Second: In LDA, we randomly assign a topic to every word in the document. Then, we try to optimize the topic by observing the data. Where is the part which is related to posterior inference in the equation above?

$\endgroup$
1
$\begingroup$

LDA is a bayesian model. The equation that you gave is the posterior distribution of the model. The alpha and beta parameters come from the fact that the dirichlet distribution, (a generalization of the beta distribution) takes these as parameters in the prior distribution. So to answer your first question, will the formula above work without the alpha and gamma, yes, you would implicitly be assuming these parameters from the prior distribution are both equal to zero. You will still get a result, but the prior you are using might be problematic on a conceptual level (maybe it would cause some numerical problems too, I'm not sure?)

To answer question 2, I want to point out that in LDA we use the conjugate prior to simplify the calculation of the optimal parameter values. That is what you are seeing in the above equation. What you are seeing is the posterior value is proportional to the left hand side. Adding the "c" terms allows you to calculate the posterior.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.