Questions tagged [topic-model]
A topic model describes text from a large corpus as a probability distribution over topics which are probability distributions over words. There are quantified contributions from all topics to a specific text.
84
questions
0
votes
0answers
11 views
Latent Dirichlet Allocation in R, topicmodels using VEM algorithm or Gibbs Sampling mixing tm and topicmodels library or WarpLDA from text2vec?
If I am trying to classify 230k text abstracts, which option would be better and more precise when aplying LDA?
1
vote
0answers
11 views
Obtain Document-Topic matrix from NMF
I recently used NMF model provided by SkLearn to obtain Topics and terms under it. SKlearn provided the following code in their documentation:
'''
Get Words in the topics
def print_top_words(model, ...
0
votes
0answers
10 views
Assign Topic to a document after LDA
I executed my LDA and now I have several topics with their word distributions.
How do I assign each document to a topic? Is Euclidean distance a good choice? Or there are other methods?
Thanks
0
votes
0answers
19 views
Same topics appearing multiple times in a NMF model
I am using NMF (Non-negative Matrix factorization) module from Scikit learn to extract 100 topics from a corpus. In contrast to LDA, the output of NMF modeling includes some of the topics multiple ...
0
votes
0answers
20 views
Probabilistic model of selecting subsets of words from documents?
Is there an existing probabilistic model that deals with the selection of subsets of words from a corpus of documents? Imagine a stack of documents where a subset of the words in each document has ...
0
votes
0answers
40 views
how to calculate coherence score in topic model
I am trying to calculate coherence score in topic modeling.
I am following this Github link
So there I need to use the preprocessed wiki and news.
I got 3 questions:
if the domain that I have ...
2
votes
0answers
23 views
Gensim LDA model: return keywords based on relevance (λ - lambda) value
I am using gensim library for topic modeling, more specifically LDA. I have created my corpus, my dictionary and my lda model, and with the help of pyLDAvis library I visualize the results. When I ...
0
votes
0answers
9 views
how to do topic modeling on very huge data?
When i come to know that gensim is useful library for topic modeling, I tried it on my huge amount of document. It works well only if the dictionary size is to be fix. In my case i have each and every ...
0
votes
0answers
9 views
0
votes
1answer
24 views
Which are the appropriate prameters for lda modeling?
I try to implement in R test for appropriate metrics for lda.
Here the way I try to use LDA
...
0
votes
0answers
54 views
Hotel Booking Analytics: Perform an analysis in order to understand the movement of the price as the day approaches the check-in date
I am working on a hotel booking dataset. I have transactional level booking data, where each row corresponds to a booking. Please refer to below snippet of the data:
I am trying to find out the ...
1
vote
1answer
42 views
Representation options of strings (keywords/topics) in models
What are all the possible ways to represent keywords in a machine learning model?
The two I am aware of are:
one hot encoding, using a static index.
vector representation, using
an embedding layer.
...
1
vote
0answers
17 views
Determining topic of text
I was wondering what I should be looking into if I want to measure the similarity between a paragraph and a corpus of text.
For example, given a paragraph of text and the entire corpus of Data ...
0
votes
0answers
24 views
Topic Modelling
New to python - topic modelling, trying to include bigrams in preprocessing
Had done the following,
...
0
votes
0answers
24 views
AWS comprehend service topic modelling takes too much time
I have an AWS Comprehand service in which i created an analysis job for topic modelling.Input to job was just 4.4kB text file, I got correct output after 25-30 minutes, then i tried with 750kB file it ...
1
vote
1answer
59 views
Find specific topics with topic modelling
I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better?
Topic Modelling ...
1
vote
1answer
20 views
Which approach to select category based on keywords
I want to assign a certain category to a group of keywords.
So i.e. people can upload images or videos, when they do this they can set keywords for this.
These keywords are free to type so words can ...
0
votes
0answers
31 views
Linking LDA topics to the input documents
I am new to LDA topic modelling. I am using gensim and am able to generate topics that make sense. Using 25k of documents, I can also print them using print_topics. ...
0
votes
1answer
27 views
Classify documents using a set of known vocabularies
I have a bunch of documents that I want to classify which ones talk about soccer (unsupervised learning, I do not want to manually label the documents).
One way I am thinking about is to go online ...
0
votes
0answers
28 views
Should my LDA topic model be skewed towards only one topic? If not then how can I un-skew it?
I'm building LDA topic models in to apply against a collection of small texts and regardless of the number of topics, I'm finding that there is always one topic that is very large (in terms of ...
1
vote
0answers
13 views
How to find out the subject of an email (in the form of a sentence) or a pdf document in NLP using Python
How to find out the subject of an email (in the form of a sentence) or a pdf document in NLP using Python. If I do topic modelling and get different groups of topic, how do I pick out the only topic ...
2
votes
1answer
230 views
Online vs Batch Learning in Latent Dirichlet Allocation using Scikit Learn
Reference: https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.LatentDirichletAllocation.html
I'm looking at the LDA algorithm from Scikit Learn for topic modeling. Can someone ...
1
vote
0answers
38 views
Perplexity calculation in variational neural topic models
I'm looking at this 2016 paper from Miao et al. https://arxiv.org/abs/1511.06038 where they use a variational autoencoder for topic modelling.
To evaluate the effectiveness of their model, they use ...
0
votes
0answers
27 views
Perform Topic Modeling By Creating a Function and Pass Along to Purrr : map() using R
My goal is to perform topic modeling (splitting into 3 topics each) by grouping words by word.trend column rather than copy and pasting block of codes for each word.trend as I have 800 unique values ...
3
votes
1answer
558 views
NLP algorithms for categorizing a list of words with specific topics
Currently I am using LDA to apply topic modeling to a corpus. Since LDA is unsupervised, it returns a set of words for a given 'topic' but doesn't necessarily specify the topic itself. I was wondering ...
0
votes
1answer
164 views
Is it correct to create topic models using both train and test data?
I have a dataset of text documents splitted into train and test sets. My task is a binary classification, classifying these documents to either 1 or -1. I have already computed some features using TF-...
0
votes
1answer
45 views
Topic Segmentation - should it be done in Raw, TfIdf or Semantic Space?
Let's assume we have a collection of documents and wish to perform some unsupervised topic segmentation.
As always, we will perform some preprocessing (including tokenization, accent-removal, ...
2
votes
1answer
798 views
How to give names/labels to topics in LDA [duplicate]
I want to give labels to different topics created using LDA. I don't want to do it manually. I saw some papers on automatic labeling but I am still confused.
How can I use the information produced ...
3
votes
1answer
245 views
What's beyond topic modeling?
I tried topic modeling (LDA, NMF) to extract insights from the data.
I'm curious right now, are there other methods for unsupervised learning to cluster documents by the same or similar context?
(...
6
votes
1answer
13k views
Resume Parsing - extracting skills from resume using Machine Learning
I am trying to extract a skill set of an employee from his/her resume. I have resumes stored as plain text in Database. I do not have predefined skills in this case. How should I approach this problem?...
3
votes
1answer
59 views
Evaluating the result of topic modeling in a way that time matters
I have run different topic modeling approach on my data(its clinical data related to Cognitive impairment diseases. we are going to process what thing is important that make it develop to more harsh ...
1
vote
1answer
36 views
How to build News Tagging model(s)
I am trying to build a news tagging system. Given a piece of news article, find 5-6 key terms from the news article that best describe the article. Refer to the image below from google news.
What are ...
1
vote
1answer
497 views
How to compare the topic coherence between models of different number of topics?
If I'm not mistaken, in this paper here http://svn.aksw.org/papers/2015/WSDM_Topic_Evaluation/public.pdf it appears that topics with larger number of topics will inherently have larger coherence ...
5
votes
1answer
151 views
How to split natural language script into segments?
I have a bunch of .txt and .srt files extracted from a MOOC website, they are the scripts of the videos. I would like to segment the scripts into parts such that each part falls into one of the ...
4
votes
1answer
314 views
Why do we need the hyperparameters beta and alpha in LDA?
I'm trying to understand the technical part of Latent Dirichlet Allocation (LDA), but I have a few questions on my mind:
First: Why do we need to add alpha and gamma every time we sample the equation ...
0
votes
2answers
286 views
Guided topic modeling: generating words from topics
I need to generate lists of words related to specific topics for a project. I am familiar with clustering methods of topic modeling such as LDA, but I have something else in mind. Are there any ...
1
vote
0answers
85 views
Performance Metric for topic extraction when there is no ground truth
I am extracting topics from text using a predefined ontology containing 2690 concepts, wordnet(to expand concept terms with their synsets, and other morphological forms of the same word) and lucene to ...
3
votes
2answers
100 views
Collection Of Variable Length Sequences and Descriptions: A Search Problem
I have a tough problem and need some advice:
Suppose I have a collection of variable length sequences, many of which are unique -- imagine the moves to a chess game, eg
d4 Nf6
c4 g6
Nc3 Bg7
...
7
votes
1answer
2k views
What is the difference between topic modeling and clustering?
I know that topic modeling and clustering are related, but not similar techniques. Can anyone suggest what are the main differences?
0
votes
1answer
377 views
Automatic question categorization when we know important words in each category
I am currently working on a question categorization problem where I automatically want to assign a category to the question. The question set I have is unlabelled. The categories for the problem are ...
1
vote
0answers
547 views
Perplexity increasing on Test DataSet in LDA (Topic Modelling)
I was plotting the perplexity values on LDA models (R) by varying topic numbers. Already train and test corpus was created.
Unfortunately, perplexity is increasing with increased number of topics on ...
1
vote
1answer
449 views
What are key dataset requirements for topic models and word embeddings?
I have a dataset of 2000 documents where avg doc size is 300 words. The vocab is dominated by domain-specific words.
My goal is to find similar documents. For this, I tried LDA, LSI, Doc2Vec (topics=...
1
vote
1answer
1k views
How to map topic to a document after topic modeling is done with LDA
Is there any way I can map generated topic from LDA to the list of documents and identify to which topic it belongs to ?
I am interested in clustering documents using unsupervised learning and ...
4
votes
2answers
936 views
Compare two topic modelling sets
I have two sets of newspaper articles where I train the first newspaper dataset separately to get the topics per each newspaper article.
...
1
vote
0answers
227 views
Hellinger Distance in Gensim
I have set of documents as follows where each document has set of words that represents the content of it.
...
4
votes
1answer
461 views
Combine two sets of clusters
I have two sets of topics obtained from two different sets of news paper articles.
In other words, Cluster_1 = ${x_1, x_2, ..., x_n}$ includes the main topics of 'X' news paper set and Cluster_2 = ${...
1
vote
1answer
236 views
Equally sized topics in Latent Dirichlet allocation
I'm using the topicmodels package for R to cluster a big set of short texts (between 10-75 words) into topics. After manually reviewing a few models it seems like there are 20 realtivly stable topics. ...
2
votes
0answers
1k views
why the accuracy of LDA model is always changing and also is high
Let’s explain the whole goal firstly, then go through the question.
I am using topic modeling like LAtent Dirichlet Allocation and NMF to extract the topic from a collection of documents.
My dataset ...
4
votes
2answers
4k views
Why we should not feed LDA with tfidf
Can someone explain why we can not feed LDA topic model with TFIDF? What is wrong with this approach conceptually?
2
votes
1answer
2k views
Apply SVM on LDA in python
hope someone kindly put time here,
my approach is like this:
TFIDF -> LDA -> SVM
I am using LDA to extract topics. I want to do topic modelling and use the topics as features to do document ...
