Questions tagged [topic-model]
A topic model describes text from a large corpus as a probability distribution over topics which are probability distributions over words. There are quantified contributions from all topics to a specific text.
118
questions
1
vote
1answer
9 views
In scikit-learn's LDA implementation, how can I sort the topics by frequency over the entire corpus?
I've used scikit-learn to perform LDA topic modeling, and I'd ultimately like to sort the topics by saliency/frequency over the entire corpus, but I'm unsure how to do as such.
I've used pyldavis ...
0
votes
0answers
20 views
NLP for recognising abstract concepts
I know that NLP algorithms can be trained to recognise the topic of a document or part of a document. I would like to train an algorithm to recognise certain abstract concepts. For example I want to ...
0
votes
0answers
18 views
LSA Model Improvement
I followed gensim's Core Tutorial and build an LSA Classification, topic modeling and Document Similarity model for newsgroups dataset.
My code is available here.
I need help with below 3 concepts.
...
1
vote
0answers
30 views
Measuring Topic-coherence score & optimal number of topics in LDA Topic Modeling in Orange data mining
I'm trying to build an Orange workflow to perform LDA topic modeling for analyzing a text corpus (.CSV dataset).
Unfortunately, the LDA widget in Orange lacks for advanced settings when comparing it ...
1
vote
0answers
16 views
Modeling of topics orthogonal to a given patterns
How to force the topics to be different from the defined ones?
Suppose I have a collection of texts about cats and dogs.There should naturally be two topics: one about dogs and one about cats. But I'm ...
2
votes
0answers
21 views
Evaluate Topic Modelling on synthetic data
I try to find the optimal number of topics on a synthetic corpus (so a list of lists of tokens I generate using various parameters). I, therefore, know the true number of topics and the true topics ...
0
votes
3answers
97 views
Best Python NLP library for supervised topic classification
I have a labeled dataset that I have ingested into a dataframe. It consists of news articles,
...
4
votes
1answer
98 views
Calculating optimal number of topics for topic modeling (LDA)
am going to do topic modeling via LDA. I run my commands to see the optimal number of topics. The output was as follows: It is a bit different from any other plots that I have ever seen. Do you think ...
2
votes
1answer
31 views
Do weights of keywords for each topic add up to 1 in topic modeling?
When you run a topic modeling (say LDA), you can get outputs for some number of topics with corresponding keywords and their weights. Based on my understanding, people usually output top 10 or top 20 ...
0
votes
1answer
30 views
Addressing polysemy in NLP tasks
Looking for modern algorithms using NN Language Model implementations addressing polysemy in NLP tasks, including text classification, question answering and topic modeling. Transfer/Zero-short ...
0
votes
0answers
42 views
Cluster images labels in some given categories using word embeddings
Given:
set of images Labels in string format each one. Also I've given a set of Categories, also in string. ($Images \neq Categories $)
Goal: I need to map given labels to given categories to "...
1
vote
1answer
31 views
What does updated alpha mean in LDA model?
I'm trying to understand LDA model by reading through implementations of the algorithm.
Many implementations update alpha during training iterations with codes like:
...
0
votes
0answers
8 views
Topic Assginment via correlation
I have an uncleaned text field something like Apple Juice xxx Newyork .. Store and an assigned topic Juice Centre.
What are ...
0
votes
0answers
178 views
How to measure the Optimal-number of Topics & Topic-Coherence score in Orange's LDA Topic-Modeling?
I'm trying to build a suitable "code-free" Topic Model in "Orange Data Mining" software for the experimental part of my Thesis, in order to analyze the themes and trends of ...
0
votes
0answers
4 views
Vector dimensionality seems to be implemented incorrectly
I'm trying to implement a fuzzy topic modeling approach in Python based on a paper, which is accommodated with an R implementation from GitHub. In one of the first steps a document term matrix is ...
0
votes
0answers
15 views
How does LDA model do inference on new documents?
Through Latent Dirichlet Allocation (LDA) one can learn topics from texts. The trained topic model can then be used on unseen data to infer to what extent each of the learned topics are represented in ...
1
vote
1answer
37 views
LDA topic model has 0-weight topics, is that normal?
While experimenting with different number of topics for the Gensim implementation of LDA, I found that for a high number of topics, the output often consists of topics with all weights equal to zero. ...
1
vote
0answers
14 views
Best measure to indicate quality of LDA model
On my corpora, I am running LDA with different settings (I experiment with different number of topics, different different ngrams and TFIDF or regular BOW).
Now, I want to rank these setups to select ...
0
votes
0answers
22 views
Extract keywords from set of similar sentences by using python
I have a list of similar sentences, and I would like to automatically extract top-n important keywords [single word length] from a whole set of those sentences by using python. These sentences are ...
1
vote
0answers
32 views
Self-supervised learning for automatic labeling of data using LDA and Word2Vec
I am trying to implement this paper A Brand-New Look at You: Predicting Brand Personality in Social Media Networks with Machine Learning for labeling Twitter data of brands with a corresponding brand ...
0
votes
1answer
30 views
0
votes
1answer
224 views
Get most likely topic per document in pandas dataframe using gensim
I am using gensim LDA to build a topic model for a bunch of documents that I have stored in a pandas data frame. Once the model is built, I can call ...
2
votes
1answer
31 views
Tweet Classification into topics- What to do with data
Good evening,
First of all, I want to apologize if the title is misleading.
I have a dataset made of around 60000 tweets, their date and time as well as the username. I need to classify them into ...
1
vote
1answer
50 views
How to compare topics generated from topic modeling from different datasets?
I have two datasets of a similar theme. Let's assume Dataset A and Dataset B. Using the top2vec model (https://github.com/ddangelov/Top2Vec) (https://arxiv.org/abs/2008.09470) on each dataset, I came ...
1
vote
1answer
60 views
Classify tweets by topic [closed]
I am approaching machine learning for the first time because of my studies. I have been given a bunch of tweets and the goal is to classify them per topic. I really have no clue on how this should be ...
0
votes
0answers
32 views
Use KL divergence to label topics from LDA
I have used sklearn's sklearn.decomposition.LatentDirichletAllocation module to model 10 topics in a set of documents.
I have also used the same on a reference text (1 document) and obtained a 1 topic ...
1
vote
1answer
32 views
Twitter Data-Analyse: What can I do with the data?
I retrieve data to a specific topic from Twitter and did my sentiment analysis on it. I never did anything in NLP, etc. So what else can I do with that? "Main goal" would be to find out if ...
1
vote
1answer
43 views
Hierarchical dirichlet process results
I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution.
The ...
0
votes
1answer
30 views
Which tool is good to collect tweets on 50 keywords over the last 5 years and then analyze them with the LDA algorithm or sentiment analysis?
I want to find tweets from the last 5 years to a topic. For this I decide for 50 Keywords (related to the main topic), where I want to find data on Twitter. I want to find out how the trend on the ...
1
vote
1answer
412 views
TF-IDF for Topic Modeling
Can TF-IDF be used a sole method for Topic Modeling ? (I know there are better methods like LDA , LSA etc)
I just want to understand if TF-IDF alone can help us in Topic modeling . If yes , can ...
3
votes
2answers
401 views
Automatic topic labelling for topic modelling
I am just curious to know if there is a way to automatically get the lables for the topics in Topic modelling. It would be really helpful if there's any python implementation of it.
1
vote
1answer
24 views
Differerent ways to detect the appropriate number of topics
I implemet the LDA topic modeling in R.
One essential parameter is the selection of the number of topics
Which of the following ways could it be the most suitable:
...
1
vote
1answer
162 views
Intuition of LDA
Can anyone explain how the LDA-topic model assigns words to topics?
I understand the generative property of the LDA model but how does the model recognize that "Labrador" and "dog" are similar words/ ...
0
votes
1answer
65 views
How to identify topic transition in consecutive sentences using Python?
I'm new to data mining. I want to detect topic transition among consecutive sentences. For instance, I have a paragraph (this could be a collection of dozens of sentences, sometimes without ...
0
votes
1answer
51 views
Topic Similarity Measure | Multi-class Text Classification Model
I am trying to build a multi class text-classifier that classifies whether the tweet belongs to one of the categories ( Advise or Science or others )
let the input be any tweet like this ,
Input :
<...
1
vote
0answers
75 views
Trying to use term document matrix as input to orange 3
I have a CSV file that has tokens as columns and documents as rows, where the rest of the cells are ints that represent term frequency. I'm trying to use this as input into Orange 3, but Orange 3 ...
1
vote
1answer
57 views
How to perform topic modelling on query search results
How can I model topics in the results returned by a search engine with higher weightage to documents ranked higher in the result set?
The use case that I am looking at involves extracting the most ...
1
vote
1answer
249 views
What does online learning mean in Topic modeling (LDA) - Gensim
I came across this line in the Gensim Documentation- Gensim LDA - "The model can also be updated with new documents for online training."
So my assumption on what it means is - 'Once we have a ...
1
vote
2answers
43 views
Reaching 100% accurray in Data Mining
I am currently working with Topic Models, especially LDA, and now I am asking myself if it's possible to reach total accurracy regarding the results.
If I insepct the results of my Topic Model, the ...
2
votes
0answers
20 views
Topic modelling on only 24 documents gives the same “topic” for any K
Description:
I have 24 documents, each one of around 2.5K tokens. They are public speeches.
My text preprocessing pipeline is a generic one, including punctuation removal, expansion of English ...
5
votes
1answer
193 views
Would Topic Modelling be classified as NLP or NLU?
I recently started my journey into the world of NLP, it's been one heck of a ride. I'm currently trying to understand whether topic modelling would be considered as NLP or NLU.
Initially I would ...
0
votes
2answers
146 views
Topic models for non-textual data?
I am looking to employ an unsupervised clustering on a dataset where each observation has a mix of textual and non-textual features.
For each observation, I combine the features into a single vector ...
1
vote
0answers
17 views
Searching for ressource recommendations (books,papers) to validate the results of a Topic Model (LDA)?
Hi I am building a Topic Model Process with Python.
To do this I am using the LDA.
However I am having trouble to determine the ideal amount of topics.
Currently I am using the CoherenceModel ...
1
vote
1answer
2k views
Datasets for Topic Modeling [closed]
I'm looking to try and use deep learning methods for topic modeling as opposed to the more traditional methods of lda and word embedding methods. However, I'm having trouble finding good labeled ...
1
vote
2answers
119 views
Clustering Small Text Descriptions
Im presented with a unique text classification problem.
Im given a list of descriptions each containing 3-8 words. I know that there are some descriptions that are nearly the same, but the majority ...
1
vote
0answers
138 views
Latent Dirichlet Allocation in R, topicmodels using VEM algorithm or Gibbs Sampling mixing tm and topicmodels library or WarpLDA from text2vec?
If I am trying to classify 230k text abstracts, which option would be better and more precise when aplying LDA?
6
votes
0answers
587 views
Gensim LDA model: return keywords based on relevance (λ - lambda) value
I am using gensim library for topic modeling, more specifically LDA. I have created my corpus, my dictionary and my lda model, and with the help of pyLDAvis library I visualize the results. When I ...
1
vote
1answer
70 views
how to do topic modeling on very huge data?
When i come to know that gensim is useful library for topic modeling, I tried it on my huge amount of document. It works well only if the dictionary size is to be fix. In my case i have each and every ...
0
votes
1answer
41 views
Which are the appropriate prameters for lda modeling?
I try to implement in R test for appropriate metrics for lda.
Here the way I try to use LDA
...
1
vote
1answer
44 views
Representation options of strings (keywords/topics) in models
What are all the possible ways to represent keywords in a machine learning model?
The two I am aware of are:
one hot encoding, using a static index.
vector representation, using
an embedding layer.
...
