40

HDP is an extension of LDA, designed to address the case where the number of mixture components (the number of "topics" in document-modeling terms) is not known a priori. So that's the reason why there's a difference. Using LDA for document modeling, one treats each "topic" as a distribution of words in some known vocabulary. For each document a mixture ...


30

Text Classification I give you a bunch of documents, each of which has a label attached. I ask you to learn why you think the contents of the documents have been given these labels based on their words. Then I give you new documents and ask what you think the label for each one should be. The labels have meaning to me, not to you necessarily. Topic ...


25

Anecdotally, I've never been impressed with the output from hierarchical LDA. It just doesn't seem to find an optimal level of granularity for choosing the number of topics. I've gotten much better results by running a few iterations of regular LDA, manually inspecting the topics it produced, deciding whether to increase or decrease the number of topics, and ...


21

Stop words are usually thought of as "the most common words in a language". However, other definitions based on different tasks are possible. It clearly makes sense to consider 'not' as a stop word if your task is based on word frequencies (e.g. tf–idf analysis for document classification). If you're concerned with the context (e.g. sentiment analysis) ...


19

I wanted to point out, since this is one of the top Google hits for this topic, that Latent Dirichlet Allocation (LDA), Hierarchical Dirichlet Processes (HDP), and hierarchical Latent Dirichlet Allocation (hLDA) are all distinct models. LDA models documents as dirichlet mixtures of a fixed number of topics- chosen as a parameter of the model by the user- ...


17

The Dirichlet distribution is a multivariate distribution. We can denote the parameters of the Dirichlet as a vector of size K of the form ~$\frac{1}{B(a)} \cdot \prod\limits_{i} x_i^{a_{i-1}}$, where $a$ is the vector of size $K$ of the parameters, and $\sum x_i = 1$. Now the LDA uses some constructs like: a document can have multiple topics (because of ...


11

But I don't know what is difference between text classification and topic models in documents Text Classification is a form of supervised learning, hence the set of possible classes are known/defined in advance, and won't change. Topic Modeling is a form of unsupervised learning (akin to clustering), so the set of possible topics are unknown apriori. They'...


11

Assuming symmetric Dirichlet distributions (for simplicity), a low alpha value places more weight on having each document composed of only a few dominant topics (whereas a high value will return many more relatively dominant topics). Similarly, a low beta value places more weight on having each topic composed of only a few dominant words.


11

I'm not sure Topic Modelling will help you here, as it tries to extract abstract topics from text. I'm afraid resumes might be too 'dry' for it to work nicely. Here are a few sources I found that might be helpful: A resume parser The reply to this post, that gives you some text mining basics (how to deal with text data, what operations to perform on it, ...


9

If you're working in R, Carson Sievert's tutorial on using LDA to model topics in movie reviews is an excellent starting point: http://cpsievert.github.io/LDAvis/reviews/reviews.html This tutorial makes use of LDAvis, an interactive visualization of topic and word distributions that can really aid intuition. Also, although not short, David M. Blei's ...


8

The purpose of topic modeling methods is to discover the latent themes (topics) assumed to have generated the documents of a corpus. Topic modeling methods are built on the distributional hypothesis, suggesting that similar words occur in similar contexts. To this end, they assume a generative process (a sequence of steps), which is a set of assumptions ...


7

I highly recommend this tutorial: Getting Started with Topic Modeling and MALLET Here are some additional links to help you get started... Good introductory materials (including links to research papers): http://www.cs.princeton.edu/~blei/topicmodeling.html Software: MALLET (Java): http://mallet.cs.umass.edu/topics.php topic modeling developer's guide: ...


7

Maybe I'm having trouble formulating the inherent difference between NLP and NLU, when do we draw the line between the two? There is a confusion here: NLP is the whole domain of AI which deals with natural language. It includes virtually any task related to processing language data (usually mostly written data, but that's not the point). Topic modeling is ...


6

I can suggest several papers on this topic: Automatic Labelling of Topic Models Automatic Labeling Hierarchical Topics Representing Topics Labels for Exploring Digital Libraries You can find more by looking at their citations.


6

To compare two LDA topics, you're really trying to compute the distance between two probability distributions. One such measure that's commonly used in these circumstances is the Hellinger Distance. To find the closest match for $x_1$ in the topics for $y$, you would calulate the Hellinger Distance between $x_1$ and each $y$ topic, then take the lowest one....


4

Topic models are usually unsupervised. There are "supervised topic models", too; but even then they try to model topics within a classes. E.g. you may have a class "football", but there may be topics inside this class that relate to particular matches or teams. The challenge with topics is that they change over time; consider the matches example above. ...


4

Try before doing LDA look at the data - like doing TF, IDF and TFIDF analysis to identify such words which happen in all subject. If You have some taxonomy of Your product definition - consider using it. In my case it was really helpful. I experimented with LDA topic modeling for recommendation systems purposes. I've few runs for offers in our marketplace ...


4

It could be partially due to the number of topics you selected but the fact that two words rank high for a given topic doesn't necessarily mean that the two words will frequently occur in the same document. Consider a topic that generally corresponds to "medicine". You could have a number of medical documents associated with heart disease and a number of ...


4

I am staying quite generic since you asked for enlightenment, just mentioning some possible directions that you can explore. You have basically two possibilities: Classification of the text (Supervised learning). Supervised means that you need first to externally apply labels (for example manually by humans) to examples of texts (labels could be "politics"...


4

To build off Mashimo's answer, one straightforward approach for topic modeling is "Latent Dirichlet Allocation" (LDA). The basic idea behind LDA is explained in this really good tutorial. Essentially, documents are assumed to be composed of mixtures of topics, which are in turn composed of mixtures of words. If we knew the topic and document distributions, ...


4

I'm not an expert in this field but you should take a look at the work of Bhargav Srinivasa Desikan, a gensim contributor, who works a lot with topic modelling. He has a couple of notebooks on his github account which could interest you, especially this one (should be pretty much your use case if I understand your problem correctly). The aforementioned ...


4

I haven't seen anything like this before but it seems quite feasible. You need an ontology to separate the main concept into its subconcepts, then you need a classifier to distinguish between your broader categories; description, methodology, classifier, application, and example. That is, I would manually label some transcripts at the paragraph level. If you ...


4

You could use doc2vec to create vector representations of each document. Once you have all the vector representations you can use standard unsupervised clustering techniques like k-means, hierarchical clustering, or K-SOM. The doc2vec model you create will be able to compute cosine similarity between two documents and also find the n most similar documents ...


3

LDA is Bayesian model. This means the desired result is a posterior probability distribution over the random vectors of interest (probability of topics etc. having seen some data). Inference for many Bayesian models is done by Markov Chain Monte Carlo. Indeed the wiki on LDA suggests that Gibbs sampling is a popular inference technique for LDA. MCMC draws ...


3

I think, even before doing LDA, you should remove words which appear in more than "x" percent of your documents. Try different "x" starting from 80% and then going down. The logic is that if the word is common for many documents, it does not distinguished those and should be neglected.


3

NMF and LDA models produce topic-word and document-topic distributions, so you can compare these models on evaluation tasks of topic coherence (i.e. evaluating topic-word distributions), document clustering/classification (i.e. evaluating document-topic distributions) or information retrieval (i.e. evaluating topic-word and document-topic distributions ...


3

Since the StackOverflow link in the question comments seems broken, here is another reply that addresses the same question: https://stackoverflow.com/a/44789327/6470915 Direct quote: In fact, Blei (who developed LDA), points out in the introduction of the paper of 2003 (entitled "Latent Dirichlet Allocation") that LDA addresses the shortcomings of the TF-...


3

It is normal, the best explantation I found for it, is from physics. Since Gibbs Sampling was known in physics long before LDA and LDA can simply be seen as a kind of matrix factorization. There is a system which has particles (words) and the particles can be in different states (topics). States with lower energy have a higher probability of being occupied ...


3

You should only use your training set in this context though it may benefit you if you used an additional cross-validation set for feature selection. If you use training data in your feature selection step you will optimistically bias your model and expose it to overfitting because of the data leakage. A similar scenario is also described in this post.


Only top voted, non community-wiki answers of a minimum length are eligible