Questions tagged [topic-model]

A topic model describes text from a large corpus as a probability distribution over topics which are probability distributions over words. There are quantified contributions from all topics to a specific text.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Calculate an ambiguity score based on topic models and Hellinger distance

I am trying to calculate some sort of ambiguity score from text based on topic probabilities from a Latent Dirichlet Allocation model and the Hellinger distance between the topic distributions. Let’s ...
user avatar
0 votes
0 answers
40 views

Are the word of women and men different when expressing their views on the same subject?

My data includes women's comments on X and Y and men's comments on X and Y. Each comment is of equal length. I will calculate how much different the word choice between men and women when commenting ...
user avatar
  • 9
0 votes
1 answer
16 views

How can I implement text classification for this problem?

Given a collection of documents - each corresponding to some economic entity - I am looking to extract information and populate a table with predetermined headings. I have a small sample of this ...
user avatar
  • 11
0 votes
0 answers
7 views

Understanding output of gensim LDA topic modeling API

I was trying to understand gensim mallet wrapper for topic modeling as explained in this notebook. In point 11, it prepares corpus which if of format Term Document frequency: ...
user avatar
  • 137
1 vote
1 answer
53 views

Topic Modelling in an existing dataframe in python

I am trying to perform topic extraction in a panda dataframe. I am using LDA topic modeling in order to extract the topics in my dataframe. No problem. But, I would like to apply LDA topic modeling ...
user avatar
0 votes
0 answers
15 views

Multi-Label Text Topic Classification

I have a huge dataset of messages/comments classified with topics. The dataset consists of 1kk records and have a total of 90 topics, like this: ...
user avatar
0 votes
1 answer
12 views

Customer Voice - Topic Modeling

I am working on a project to classify some messages received from our customers. Basically I have to get the main problem of those messages (hundreds of messages are received every day). Our SAC team ...
user avatar
  • 1
0 votes
0 answers
26 views

Grouping tweets and newspaper articles by topic

i want to implement a software that groups twitter posts to other twitter posts or to newspaper articles with similar topics. Let's say for example someone tweets about a soccer game and at the same ...
user avatar
  • 1
2 votes
2 answers
53 views

Topic Modeling: LDA vs LSA vs ToPMine

I am new to Topic Modeling. Is it possible to implement ToPMine in Python? In a quick search, I can't seem to find any Python package with ToPMine. Is ToPMine better than LDA and LSA? I am aware ...
user avatar
  • 159
1 vote
0 answers
15 views

Two sets of topics/words in Topic Modeling

In short, the question is: I have two sets of words per document. I would like to extract two sets of topics per document corresponding to sets of words. To be more precise: Document(d) can be ...
user avatar
  • 11
0 votes
0 answers
13 views

Browser history segmentation

I'm trying to segment a browser history into semantically coherent sessions. For example, a user might be working on a school project for 30 minutes, then planning an upcoming vacation for 30 minutes, ...
user avatar
2 votes
1 answer
68 views

How does amazon's reviews that mention extracts topics from reviews?

Amazon product page contains a section called Reviews that mention. The section lists the main things that users liked or dislike about the product. For example see ...
user avatar
1 vote
0 answers
43 views

Creating a Sentiment dictionary from scratch

I am analyzing Arabic textual data from a social media forum discussing economic issues such as labor unions. I am using a package that classifies as negative, positive, or neutral. For instance, the ...
user avatar
1 vote
1 answer
31 views

Latent Dirichlet Allocation (LDA) importance of document generation and Gibbs Sampling

I am having trouble finding the correlation between the two seemingly uncorrelated parts of LDA. What I understood from several videos is: There is a document generation "part", which is ...
user avatar
  • 23
1 vote
1 answer
58 views

Updating a genism LDA model with new documents and topics

I have a conceptual problem that is related to a project I'm working on. I'm relatively new to the domain of NLP so this might be a poor question but I would really appreciate any help. My dataset is ...
user avatar
1 vote
1 answer
40 views

Topic modelling on long documents: intra document clustering first

I have a collection (around 1000) of very noisy, similar documents, that are each very long (>10 pages - 600 paragraphs) with multiple subsections - I want to perform topic modelling across the ...
user avatar
1 vote
1 answer
49 views

Diachronic topic modeling with chaning set of topics

In short, the question is: how can I build a regularly updated chain of topics which would also show how topics emerge and disappear over time? To be more precise: I have a data with timestamps ...
user avatar
  • 11
0 votes
0 answers
12 views

Understanding the usage of variational methods in LDA

I'm reading a research paper on LDA. My goal is to understand the difference between variational methods and sampling algorithms. The specific article review that I am reading is: I would like help ...
user avatar
0 votes
0 answers
14 views

NLP Data Reduction With Similarity Detection

I'm trying to use natural language processing to categorize transaction data. Ultimately I want to create labels for transactions (topic modeling). For example: Transaction Description Label "...
user avatar
  • 101
1 vote
1 answer
23 views

Choice of the number of topics (clusters) in textual data

I have a social science background and I'm doing a text mining project. I'm looking for advice about the choice of the number of topics/clusters when analyzing textual data. In particular, I'm ...
user avatar
  • 11
0 votes
2 answers
136 views

Measuring coherence score for Top2Vec models

I am working on creating a number of Top2Vec models on Reddit threads. I am basically changing the HDBScan cluster sizes to get different clusters of the Doc2Vec embeddings representing a different # ...
user avatar
  • 1
0 votes
0 answers
25 views

How to interpret different coherence values

For an experiment with topic models, I have calculated four coherence values using Gensim's implementation: c_v u_mass c_uci c_npmi From this paper, I know that c_v correlates mostly with human ...
user avatar
  • 189
1 vote
1 answer
42 views

In scikit-learn's LDA implementation, how can I sort the topics by frequency over the entire corpus?

I've used scikit-learn to perform LDA topic modeling, and I'd ultimately like to sort the topics by saliency/frequency over the entire corpus, but I'm unsure how to do as such. I've used pyldavis ...
user avatar
  • 113
0 votes
0 answers
30 views

NLP for recognising abstract concepts

I know that NLP algorithms can be trained to recognise the topic of a document or part of a document. I would like to train an algorithm to recognise certain abstract concepts. For example I want to ...
user avatar
  • 101
0 votes
0 answers
28 views

LSA Model Improvement

I followed gensim's Core Tutorial and build an LSA Classification, topic modeling and Document Similarity model for newsgroups dataset. My code is available here. I need help with below 3 concepts. ...
user avatar
  • 1
1 vote
0 answers
100 views

Measuring Topic-coherence score & optimal number of topics in LDA Topic Modeling in Orange data mining

I'm trying to build an Orange workflow to perform LDA topic modeling for analyzing a text corpus (.CSV dataset). Unfortunately, the LDA widget in Orange lacks for advanced settings when comparing it ...
user avatar
1 vote
0 answers
22 views

Modeling of topics orthogonal to a given patterns

How to force the topics to be different from the defined ones? Suppose I have a collection of texts about cats and dogs.There should naturally be two topics: one about dogs and one about cats. But I'm ...
user avatar
  • 111
2 votes
0 answers
22 views

Evaluate Topic Modelling on synthetic data

I try to find the optimal number of topics on a synthetic corpus (so a list of lists of tokens I generate using various parameters). I, therefore, know the true number of topics and the true topics ...
user avatar
0 votes
3 answers
289 views

Best Python NLP library for supervised topic classification

I have a labeled dataset that I have ingested into a dataframe. It consists of news articles, ...
user avatar
4 votes
1 answer
338 views

Calculating optimal number of topics for topic modeling (LDA)

am going to do topic modeling via LDA. I run my commands to see the optimal number of topics. The output was as follows: It is a bit different from any other plots that I have ever seen. Do you think ...
user avatar
2 votes
1 answer
43 views

Do weights of keywords for each topic add up to 1 in topic modeling?

When you run a topic modeling (say LDA), you can get outputs for some number of topics with corresponding keywords and their weights. Based on my understanding, people usually output top 10 or top 20 ...
user avatar
  • 123
0 votes
1 answer
75 views

Addressing polysemy in NLP tasks

Looking for modern algorithms using NN Language Model implementations addressing polysemy in NLP tasks, including text classification, question answering and topic modeling. Transfer/Zero-short ...
user avatar
  • 263
1 vote
1 answer
61 views

What does updated alpha mean in LDA model?

I'm trying to understand LDA model by reading through implementations of the algorithm. Many implementations update alpha during training iterations with codes like: ...
user avatar
1 vote
1 answer
116 views

LDA topic model has 0-weight topics, is that normal?

While experimenting with different number of topics for the Gensim implementation of LDA, I found that for a high number of topics, the output often consists of topics with all weights equal to zero. ...
user avatar
  • 189
1 vote
0 answers
22 views

Best measure to indicate quality of LDA model

On my corpora, I am running LDA with different settings (I experiment with different number of topics, different different ngrams and TFIDF or regular BOW). Now, I want to rank these setups to select ...
user avatar
  • 189
1 vote
0 answers
40 views

Self-supervised learning for automatic labeling of data using LDA and Word2Vec

I am trying to implement this paper A Brand-New Look at You: Predicting Brand Personality in Social Media Networks with Machine Learning for labeling Twitter data of brands with a corresponding brand ...
user avatar
  • 213
0 votes
1 answer
132 views

Topic modelling with many synonyms - how to extract 'latent themes'

Here's my corpus ...
user avatar
  • 131
0 votes
1 answer
708 views

Get most likely topic per document in pandas dataframe using gensim

I am using gensim LDA to build a topic model for a bunch of documents that I have stored in a pandas data frame. Once the model is built, I can call ...
user avatar
  • 167
2 votes
1 answer
41 views

Tweet Classification into topics- What to do with data

Good evening, First of all, I want to apologize if the title is misleading. I have a dataset made of around 60000 tweets, their date and time as well as the username. I need to classify them into ...
user avatar
1 vote
1 answer
136 views

How to compare topics generated from topic modeling from different datasets?

I have two datasets of a similar theme. Let's assume Dataset A and Dataset B. Using the top2vec model (https://github.com/ddangelov/Top2Vec) (https://arxiv.org/abs/2008.09470) on each dataset, I came ...
user avatar
1 vote
1 answer
91 views

Classify tweets by topic [closed]

I am approaching machine learning for the first time because of my studies. I have been given a bunch of tweets and the goal is to classify them per topic. I really have no clue on how this should be ...
user avatar
1 vote
1 answer
42 views

Twitter Data-Analyse: What can I do with the data?

I retrieve data to a specific topic from Twitter and did my sentiment analysis on it. I never did anything in NLP, etc. So what else can I do with that? "Main goal" would be to find out if ...
user avatar
  • 27
1 vote
1 answer
47 views

Hierarchical dirichlet process results

I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution. The ...
user avatar
0 votes
1 answer
38 views

Which tool is good to collect tweets on 50 keywords over the last 5 years and then analyze them with the LDA algorithm or sentiment analysis?

I want to find tweets from the last 5 years to a topic. For this I decide for 50 Keywords (related to the main topic), where I want to find data on Twitter. I want to find out how the trend on the ...
user avatar
  • 27
1 vote
1 answer
1k views

TF-IDF for Topic Modeling

Can TF-IDF be used a sole method for Topic Modeling ? (I know there are better methods like LDA , LSA etc) I just want to understand if TF-IDF alone can help us in Topic modeling . If yes , can ...
user avatar
  • 157
3 votes
2 answers
649 views

Automatic topic labelling for topic modelling

I am just curious to know if there is a way to automatically get the lables for the topics in Topic modelling. It would be really helpful if there's any python implementation of it.
user avatar
1 vote
1 answer
36 views

Differerent ways to detect the appropriate number of topics

I implemet the LDA topic modeling in R. One essential parameter is the selection of the number of topics Which of the following ways could it be the most suitable: ...
user avatar
  • 147
1 vote
1 answer
264 views

Intuition of LDA

Can anyone explain how the LDA-topic model assigns words to topics? I understand the generative property of the LDA model but how does the model recognize that "Labrador" and "dog" are similar words/ ...
user avatar
0 votes
1 answer
138 views

How to identify topic transition in consecutive sentences using Python?

I'm new to data mining. I want to detect topic transition among consecutive sentences. For instance, I have a paragraph (this could be a collection of dozens of sentences, sometimes without ...
user avatar
  • 111
0 votes
1 answer
59 views

Topic Similarity Measure | Multi-class Text Classification Model

I am trying to build a multi class text-classifier that classifies whether the tweet belongs to one of the categories ( Advise or Science or others ) let the input be any tweet like this , Input : <...
user avatar
  • 123