Questions tagged [topic-model]

A topic model describes text from a large corpus as a probability distribution over topics which are probability distributions over words. There are quantified contributions from all topics to a specific text.

Filter by
Sorted by
Tagged with
2
votes
1answer
270 views

How to get columns from unsorted rows in Pandas? (MALLET)

My data (the doc-topics output from a MALLET topic model) has the following shape: ...
-1
votes
1answer
131 views

Topic modelling or simple case of calculating probabilities?

I am trying to find the common topics between articles read using the respective tags attached to each article. Background of my mini project: The problem I am trying to solve involves looking at ...
1
vote
0answers
82 views

automated topic modeling topic naming [duplicate]

Are there well-known automated methods for deriving a name for each topic obtained through topic modeling? for a specifically given problem at hand I will probably default to an algorithm on top an ...
1
vote
1answer
109 views

Is what I did supervised or unsupervised Machine learning?

My goal is to get a smartphone names from Twitter. So this is what I followed: 1- I extracted 100K tweets using the keyword “smartphone”. 2- I Applied LDA after applying ngram tokenization and ...
4
votes
1answer
2k views

How can I run Labeled LDA over one textual document?

I have 200K tweets and I already a applied the LDA (Latent Dirichlet Allocation) algorithm using Gensim python library. And now I need to apply over them the labeled/supervised LDA. Can any one help ...
1
vote
1answer
2k views

Real time topic identification of news article

Let's say I'm constantly harvesting all the news article that are being published online (only having basic info about each one, eg. title, content, language, source (which news site)). Let's say ...
4
votes
3answers
152 views

How can I discover topics in a social media data-set?

I'm working on a project and i need to discover topics existing in a social media data set. For instance, i wanna extract the topics existing on 200K tweets. Any one recommend to me any machine ...
-1
votes
1answer
670 views

Using Topic Models in R

I am learning about Probabilistic Topic Models by reading this article by D. Blei, watching this video, and doing this exercise A Gentle Introduction to Topic Modeling in R. After the topics in my ...
1
vote
2answers
457 views

Memory error - Hierarchical Dirichlet Process, HDP gensim

I am running Hierarchical Dirichlet Process, HDP using gensim in Python but as my corpus is too large it is throwing me following error: ...
2
votes
0answers
125 views

Extending Author-Topic LDA

I've been trying to extend the LDA and wanted some help, direction and insight. Can Author-Topic LDA be used as a document "category" model? The premise of the Author-Topic model is that multiple ...
2
votes
1answer
407 views

What metrics must i use in my data(unstructured) preprocessing research?

i am currently working on preprocessing unstructured data (emails,logs,bug reports and irc chats). i wish to prove preprocessing improves the content quality. are there metrics available to prove ...
6
votes
4answers
6k views

How to give name to topics created using LDA?

I have categorized 800,000 documents into 500 categories using the Mahout topic modelling. Instead of representing the topic using the top 5/10 words for each topics, I want to infer a generic name ...
2
votes
3answers
2k views

replicability / reproducibility in topic modeling (LDA)

If I'm not wrong, topic modeling (LDA) is not replicable, i.e. it gives different results in different runs. Where does this come from (where does this randomness come from and why is it necessary?) ...
2
votes
2answers
540 views

How to discard trash topics from topic models?

I am undertaking text analysis of some twitter data. In the end I want to have a data that is interpretable. And so in the end I would like to reduce the data to relevant unit of analysis. Topic ...
5
votes
2answers
371 views

Why do my Latent Dirichlet Allocation Topics mix words that never co-occurred?

I have one corpus of documents on diabetes, another on Leonardo da Vinci, and another on animation and computer graphics. I combined all of these documents into a LDA and got a topic like the one ...
4
votes
2answers
378 views

Topic models for Relevance Prediction

Suppose I have data in the form of Query/Document Pairs, along with corresponding relevance scores (or class labels). Is there a way to use topic modeling to devise a model so that later given a ...
5
votes
3answers
2k views

Need help with LDA for selecting features

I am currently selecting features of products by using LDA to group 6000 keywords of product into topics. Here is the sample of my dataset after being organized into list of keywords for each product ...
2
votes
1answer
206 views

Reasons and prevention of trivial (and less trivial) misclassification errors?

I was not sure about posting this question with mentioning the name of the company, which I quite respect and admire. However, I've figured that a wider exposure might help the team to fix this and ...
9
votes
5answers
11k views

Tutorials on topic models and LDA

I would like to know if you people have some good tutorials (fast and straightforward) about topic models and LDA, teaching intuitively how to set some parameters, what they mean and if possible, with ...
28
votes
3answers
20k views

What is difference between text classification and topic models?

I know the difference between clustering and classification in machine learning, but I don't understand the difference between text classification and topic modeling for documents. Can I use topic ...
22
votes
2answers
30k views

What does the alpha and beta hyperparameters contribute to in Latent Dirichlet allocation?

LDA has two hyperparameters, tuning them changes the induced topics. What does the alpha and beta hyperparameters contribute to LDA? How does the topic change if one or the other hyperparameters ...
59
votes
6answers
27k views

Latent Dirichlet Allocation vs Hierarchical Dirichlet Process

Latent Dirichlet Allocation (LDA) and Hierarchical Dirichlet Process (HDP) are both topic modeling processes. The major difference is LDA requires the specification of the number of topics, and HDP ...

1 2
3