Skip to main content

Questions tagged [topic-model]

A topic model describes text from a large corpus as a probability distribution over topics which are probability distributions over words. There are quantified contributions from all topics to a specific text.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

Choosing the right number of tokens in a dictionary for an LDA topic modelling analysis

I have a data set of customers with n complaints. As a result, I want to perform topic modelling to find topics that customers talk about in their complaints. For this I use LDA from spacy. I have ...
methus's user avatar
  • 111
0 votes
0 answers
7 views

Finding good parameters for topic modeling using LDA and spacy

I have a data set of customers with n complaints [c_1,...,c_n]. As a result, I want to perform topic modelling to find topics that customers talk about in their complaints. For this I use LDA from ...
methus's user avatar
  • 111
0 votes
0 answers
6 views

Semantic grouping and replacement of words to improve topic modelling with LDA

I have a data set of customers with complaints. As a result, I want to perform topic modelling to find topics that customers talk about in their complaints. I use LDA for this. In the results of LDA, ...
methus's user avatar
  • 111
0 votes
0 answers
28 views

Interpreting Perplexity, U_mass coherence and Cv score trends for a Latent Dirichlet Allocation Model

I'm running an LDA model through gensim. To my understanding, closer the u_mass coherence score is to zero, higher is the interpretability of the topics that come up. I'm getting the u_mass coherence ...
Sarthak's user avatar
1 vote
1 answer
21 views

Topic modeling evaluation

I'm working on topic modeling and I have generated clusters with two different methods. How can I evaluate which method performs better than the other?
user5520049's user avatar
0 votes
1 answer
42 views

Visualizing Author Topic Similarities: t-SNE and Cluster Labeling

I am working on a dataframe containing abstracts from various NLP conferences, along with information on information on the respective authors (names) and the keywords they've associated with their ...
Marrluxia's user avatar
0 votes
0 answers
60 views

How to use Bertweet model for topic modeling

The problem is implementation of Bertweet in a topic-modeling project with understandable output like BERTopic, i want to use it on a relatively large (20k tweets) unlabelled dataset to segment it ...
Rossin's user avatar
  • 1
0 votes
0 answers
101 views

a way to automatically split a video into chapters?

Given a video with audio, we can use ASR to get a script of the sentences and timestamps. We are looking for a way to group the sentences into chapters. There are several companies that are doing it ...
Lior's user avatar
  • 101
-1 votes
1 answer
33 views

Build a topic model without data?

I need to come up with a topic model, without any labelled dataset, the model should also be multilingual, thinking of using LLM's as they are accurate and awesome but if Im to build one on my own how ...
emily 's user avatar
  • 35
0 votes
0 answers
38 views

number of topics in LDA model (coherence)

I want to fit a LDA model using RStudio, but there are some trouble in the determination of topics number. Perplexity and coherence are suggested to do this work, so first I use LDA and perplexity ...
杨昊天's user avatar
0 votes
0 answers
9 views

How to recreate a WE1S project?

The WE1S (WhatEvery1Says) project is so resourceful and well documented that I really want to use it. Unfortunately, I still don't know where is the repo that the Workspace documentation referring to? ...
Ooker's user avatar
  • 123
0 votes
0 answers
7 views

Is colorring each document and word as monochromatic as possible the goal of LDA in specific or all topic models in general?

In the video Training Latent Dirichlet Allocation: Gibbs Sampling, the Goal section at 7:32, the video says that: Goal: Color each word with blue, green, red Each article is as monochromatic as ...
Ooker's user avatar
  • 123
0 votes
0 answers
17 views

What to do when there is a jargon that is the same with a common word?

Let's say in a particular field, the word the has a specific meaning and not just be a determination. The common the one and the ...
Ooker's user avatar
  • 123
0 votes
0 answers
25 views

Identifying topic coverage from text

I have been trying to understand a feasible approach to identify if given text talks about one of the given topics(also provided as text descriptions). for e.g. ...
Rakesh K's user avatar
0 votes
1 answer
523 views

BERTopic: Is it okay to ignore the first two topics?

I used BERTopic to generate a topic model over a large dataset of texts. The result is very appealing and the modeled topics are mostly perfectly interpretable for a human, especially compared to ...
oberbus's user avatar
0 votes
2 answers
501 views

How to perform topic reduction?

I am using top2vec to perform topic modelling. According to the paper, topic reduction can be performed on the topic vectors to hierarchically group similar topics and reduce the number of topics ...
Alberto De Benedittis's user avatar
1 vote
0 answers
197 views

Short Text Topic Modelling in Python

I have a large dataset of short reviews and I would like to find the most recurring themes. For this reason, I got into topic modeling. I am looking for some good tutorials and references for short ...
Alberto De Benedittis's user avatar
1 vote
1 answer
388 views

BERTopic Visualization

I new to topic modeling and I'm trying to use BERTopic inside of PyCharm. I'm struggling to ...
Life is complex's user avatar
1 vote
2 answers
71 views

Topic Modeling - n-grams or 1,2,3,...n-grams?

Do people use n-grams or 1,2,3,...n-grams in both matrix factorisation and generative models in Topic Modeling? I've been trying to understand the basics of Topic Modeling and came to know that there ...
rahuladwani's user avatar
0 votes
1 answer
112 views

Gensim: create a dictionary from a large corpus without loading it in RAM?

The topic modelling library Gensim offers the ability to stream a large document instead of storing it in memory. Streaming is possible for the stage of converting the corpus to BOW, but the ...
Erwan's user avatar
  • 25.4k
0 votes
1 answer
42 views

How to use Fuzzy Topic Model as a Classification Model Input

I have fuzzy clustering for Topic modelling and got this . There are all total 50 topics[0 to 49] and each topic consists 30 words with a probability multiplicative factor. Now how do I make it as a ...
Amartya's user avatar
  • 133
0 votes
1 answer
195 views

Lost human names after 'Lemmatization' for topic modeling in python

I'm using gensim in Python for topic modeling. Currently, I have one problem. If I don't lemmatize, human names will appear as 'Most Relevant Terms for Topic,' but after lemmatization, the human names ...
SEan1820's user avatar
0 votes
0 answers
150 views

How can I build and train mode for Arabic word embedding from scratch using BERT and share the model on hugging face?

my project is (building an Arabic word embedding model). I want to build my own model on hugging face like (aubmindlab/AraBERT model) for Arabic language using Bert for word embedding. How can I start ...
Ali A. Jalil's user avatar
0 votes
1 answer
616 views

Topic modelling or Keyword extraction for a small dataset

I am working on a project where I have a dataset which contains very less data. These are the comments of people. I have only 130 lines with 10 words per line. My goal is to identify the common topics ...
ASHUTOSH MITRA's user avatar
1 vote
1 answer
97 views

document similarity using LDA probabilities

Let us say I have a LDA model trained on a corpus of text. I would like to know, for a newly given document, which one from the corpus is closet to it. But, to do so, I want to use probabilities ...
aRedDish's user avatar
0 votes
0 answers
151 views

List of words cluster by topics

I have a list of words, these words correspond to labels in news and they are not duplicated. I would like to get a clustering for this list based on topics. I try with wordnet but I don't know how ...
mathprogram's user avatar
0 votes
0 answers
44 views

Are the word of women and men different when expressing their views on the same subject?

My data includes women's comments on X and Y and men's comments on X and Y. Each comment is of equal length. I will calculate how much different the word choice between men and women when commenting ...
nem0's user avatar
  • 9
0 votes
1 answer
28 views

How can I implement text classification for this problem?

Given a collection of documents - each corresponding to some economic entity - I am looking to extract information and populate a table with predetermined headings. I have a small sample of this ...
Usama's user avatar
  • 11
1 vote
1 answer
2k views

Topic Modelling in an existing dataframe in python

I am trying to perform topic extraction in a panda dataframe. I am using LDA topic modeling in order to extract the topics in my dataframe. No problem. But, I would like to apply LDA topic modeling ...
Aliya Leigh's user avatar
0 votes
0 answers
61 views

Grouping tweets and newspaper articles by topic

i want to implement a software that groups twitter posts to other twitter posts or to newspaper articles with similar topics. Let's say for example someone tweets about a soccer game and at the same ...
Yanek's user avatar
  • 1
2 votes
2 answers
833 views

Topic Modeling: LDA vs LSA vs ToPMine

I am new to Topic Modeling. Is it possible to implement ToPMine in Python? In a quick search, I can't seem to find any Python package with ToPMine. Is ToPMine better than LDA and LSA? I am aware ...
Peter's user avatar
  • 179
1 vote
0 answers
21 views

Two sets of topics/words in Topic Modeling

In short, the question is: I have two sets of words per document. I would like to extract two sets of topics per document corresponding to sets of words. To be more precise: Document(d) can be ...
aloskam's user avatar
  • 11
2 votes
1 answer
219 views

How does amazon's reviews that mention extracts topics from reviews?

Amazon product page contains a section called Reviews that mention. The section lists the main things that users liked or dislike about the product. For example see ...
user2301346's user avatar
2 votes
0 answers
312 views

Creating a Sentiment dictionary from scratch

I am analyzing Arabic textual data from a social media forum discussing economic issues such as labor unions. I am using a package that classifies as negative, positive, or neutral. For instance, the ...
maldini425's user avatar
1 vote
1 answer
95 views

Latent Dirichlet Allocation (LDA) importance of document generation and Gibbs Sampling

I am having trouble finding the correlation between the two seemingly uncorrelated parts of LDA. What I understood from several videos is: There is a document generation "part", which is ...
Arik's user avatar
  • 23
1 vote
1 answer
604 views

Updating a genism LDA model with new documents and topics

I have a conceptual problem that is related to a project I'm working on. I'm relatively new to the domain of NLP so this might be a poor question but I would really appreciate any help. My dataset is ...
Tanay Roman's user avatar
1 vote
2 answers
136 views

Topic modelling on long documents: intra document clustering first

I have a collection (around 1000) of very noisy, similar documents, that are each very long (>10 pages - 600 paragraphs) with multiple subsections - I want to perform topic modelling across the ...
James Stirling's user avatar
1 vote
1 answer
166 views

Diachronic topic modeling with chaning set of topics

In short, the question is: how can I build a regularly updated chain of topics which would also show how topics emerge and disappear over time? To be more precise: I have a data with timestamps ...
yys's user avatar
  • 11
1 vote
1 answer
128 views

Choice of the number of topics (clusters) in textual data

I have a social science background and I'm doing a text mining project. I'm looking for advice about the choice of the number of topics/clusters when analyzing textual data. In particular, I'm ...
Himan's user avatar
  • 11
0 votes
2 answers
994 views

Measuring coherence score for Top2Vec models

I am working on creating a number of Top2Vec models on Reddit threads. I am basically changing the HDBScan cluster sizes to get different clusters of the Doc2Vec embeddings representing a different # ...
Teefs's user avatar
  • 1
1 vote
1 answer
405 views

In scikit-learn's LDA implementation, how can I sort the topics by frequency over the entire corpus?

I've used scikit-learn to perform LDA topic modeling, and I'd ultimately like to sort the topics by saliency/frequency over the entire corpus, but I'm unsure how to do as such. I've used pyldavis ...
Basht0n's user avatar
  • 113
0 votes
0 answers
35 views

LSA Model Improvement

I followed gensim's Core Tutorial and build an LSA Classification, topic modeling and Document Similarity model for newsgroups dataset. My code is available here. I need help with below 3 concepts. ...
Bala's user avatar
  • 1
1 vote
0 answers
366 views

Measuring Topic-coherence score & optimal number of topics in LDA Topic Modeling in Orange data mining

I'm trying to build an Orange workflow to perform LDA topic modeling for analyzing a text corpus (.CSV dataset). Unfortunately, the LDA widget in Orange lacks for advanced settings when comparing it ...
Salah Eddin's user avatar
1 vote
0 answers
33 views

Modeling of topics orthogonal to a given patterns

How to force the topics to be different from the defined ones? Suppose I have a collection of texts about cats and dogs.There should naturally be two topics: one about dogs and one about cats. But I'm ...
mc2's user avatar
  • 111
2 votes
0 answers
47 views

Evaluate Topic Modelling on synthetic data

I try to find the optimal number of topics on a synthetic corpus (so a list of lists of tokens I generate using various parameters). I, therefore, know the true number of topics and the true topics ...
seaslug95's user avatar
0 votes
3 answers
671 views

Best Python NLP library for supervised topic classification

I have a labeled dataset that I have ingested into a dataframe. It consists of news articles, ...
DrakeMurdoch's user avatar
5 votes
1 answer
1k views

Calculating optimal number of topics for topic modeling (LDA)

am going to do topic modeling via LDA. I run my commands to see the optimal number of topics. The output was as follows: It is a bit different from any other plots that I have ever seen. Do you think ...
Tahereh Maghsoudi's user avatar
2 votes
1 answer
346 views

Do weights of keywords for each topic add up to 1 in topic modeling?

When you run a topic modeling (say LDA), you can get outputs for some number of topics with corresponding keywords and their weights. Based on my understanding, people usually output top 10 or top 20 ...
Todd's user avatar
  • 123
0 votes
1 answer
220 views

Addressing polysemy in NLP tasks

Looking for modern algorithms using NN Language Model implementations addressing polysemy in NLP tasks, including text classification, question answering and topic modeling. Transfer/Zero-short ...
dokondr's user avatar
  • 295
1 vote
1 answer
339 views

What does updated alpha mean in LDA model?

I'm trying to understand LDA model by reading through implementations of the algorithm. Many implementations update alpha during training iterations with codes like: ...
user115919's user avatar