1
$\begingroup$

I'm trying to understand LDA model by reading through implementations of the algorithm.

Many implementations update alpha during training iterations with codes like:

psi_sum1 = 0.0e0;
psi_sum2 = 0.0e0;

for (int d = 0; d < doc_size; d++)
{
    psi_sum1 += digamma(float(n_dk[d][k]) + alpha_old[k]) - digamma(alpha[k]);
    psi_sum2 += digamma(float(doc_words[d]) + alpha_sum) - digamma(alpha_sum);
}

alpha[k] = alpha_old[k] * psi_sum1 / psi_sum2;

e.g. for 10 topics, the init alpha could be [1, 1, ..., 1] (length = 10), after training, the alpha is updated to [0.29, 0.30, ..., 0.28] (length = 10).

What does this updated alpha mean?

Can I use the updated alpha to determine whether the topic number is appropriate?

$\endgroup$
1
$\begingroup$

The alpha is a hyperparameter which controls the mixture of topics for any given document. Turn it down and the documents will likely have less of a mixture of topics. Turn it up and the documents will likely have more of a mixture of topics.

Can I use the updated alpha to determine whether the topic number is appropriate?

Tuning it depends upon your problem statement which could be solved with help of domain knowledge.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.