Questions tagged [topic-model]
A topic model describes text from a large corpus as a probability distribution over topics which are probability distributions over words. There are quantified contributions from all topics to a specific text.
94
questions
0
votes
1answer
13 views
Twitter Data-Analyse: What can I do with the data?
I retrieve data to a specific topic from Twitter and did my sentiment analysis on it. I never did anything in NLP, etc. So what else can I do with that? "Main goal" would be to find out if ...
1
vote
1answer
33 views
Hierarchical dirichlet process results
I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution.
The ...
0
votes
1answer
20 views
Which tool is good to collect tweets on 50 keywords over the last 5 years and then analyze them with the LDA algorithm or sentiment analysis?
I want to find tweets from the last 5 years to a topic. For this I decide for 50 Keywords (related to the main topic), where I want to find data on Twitter. I want to find out how the trend on the ...
1
vote
1answer
66 views
TF-IDF for Topic Modeling
Can TF-IDF be used a sole method for Topic Modeling ? (I know there are better methods like LDA , LSA etc)
I just want to understand if TF-IDF alone can help us in Topic modeling . If yes , can ...
2
votes
1answer
85 views
Automatic topic labelling for topic modelling
I am just curious to know if there is a way to automatically get the lables for the topics in Topic modelling. It would be really helpful if there's any python implementation of it.
1
vote
1answer
15 views
Differerent ways to detect the appropriate number of topics
I implemet the LDA topic modeling in R.
One essential parameter is the selection of the number of topics
Which of the following ways could it be the most suitable:
...
0
votes
0answers
10 views
How to improve text classification using topic modeling feature vector?
I have a binary text classification problem. I am using the topic model(LDA) trained on Wikipedia to get the feature vectors to classify documents. I have tried Logistic Regression, Random Forest, ...
1
vote
1answer
75 views
Intuition of LDA
Can anyone explain how the LDA-topic model assigns words to topics?
I understand the generative property of the LDA model but how does the model recognize that "Labrador" and "dog" are similar words/ ...
0
votes
1answer
33 views
How to identify topic transition in consecutive sentences using Python?
I'm new to data mining. I want to detect topic transition among consecutive sentences. For instance, I have a paragraph (this could be a collection of dozens of sentences, sometimes without ...
0
votes
1answer
30 views
Topic Similarity Measure | Multi-class Text Classification Model
I am trying to build a multi class text-classifier that classifies whether the tweet belongs to one of the categories ( Advise or Science or others )
let the input be any tweet like this ,
Input :
<...
0
votes
0answers
54 views
Trying to use term document matrix as input to orange 3
I have a CSV file that has tokens as columns and documents as rows, where the rest of the cells are ints that represent term frequency. I'm trying to use this as input into Orange 3, but Orange 3 ...
1
vote
1answer
22 views
How to perform topic modelling on query search results
How can I model topics in the results returned by a search engine with higher weightage to documents ranked higher in the result set?
The use case that I am looking at involves extracting the most ...
0
votes
0answers
60 views
Topic Modelling LDA - Small dataset
I'm trying to analyze a corpus of 100 texts with the topic modelling widget of Orange using LDA algorithm. When I increase the number of topics, they repeat themselves so I have in output some ...
0
votes
1answer
85 views
What does online learning mean in Topic modeling (LDA) - Gensim
I came across this line in the Gensim Documentation- Gensim LDA - "The model can also be updated with new documents for online training."
So my assumption on what it means is - 'Once we have a ...
1
vote
2answers
39 views
Reaching 100% accurray in Data Mining
I am currently working with Topic Models, especially LDA, and now I am asking myself if it's possible to reach total accurracy regarding the results.
If I insepct the results of my Topic Model, the ...
2
votes
0answers
16 views
Topic modelling on only 24 documents gives the same “topic” for any K
Description:
I have 24 documents, each one of around 2.5K tokens. They are public speeches.
My text preprocessing pipeline is a generic one, including punctuation removal, expansion of English ...
5
votes
1answer
153 views
Would Topic Modelling be classified as NLP or NLU?
I recently started my journey into the world of NLP, it's been one heck of a ride. I'm currently trying to understand whether topic modelling would be considered as NLP or NLU.
Initially I would ...
0
votes
2answers
67 views
Topic models for non-textual data?
I am looking to employ an unsupervised clustering on a dataset where each observation has a mix of textual and non-textual features.
For each observation, I combine the features into a single vector ...
1
vote
0answers
17 views
Searching for ressource recommendations (books,papers) to validate the results of a Topic Model (LDA)?
Hi I am building a Topic Model Process with Python.
To do this I am using the LDA.
However I am having trouble to determine the ideal amount of topics.
Currently I am using the CoherenceModel ...
1
vote
1answer
911 views
Datasets for Topic Modeling [closed]
I'm looking to try and use deep learning methods for topic modeling as opposed to the more traditional methods of lda and word embedding methods. However, I'm having trouble finding good labeled ...
1
vote
2answers
68 views
Clustering Small Text Descriptions
Im presented with a unique text classification problem.
Im given a list of descriptions each containing 3-8 words. I know that there are some descriptions that are nearly the same, but the majority ...
0
votes
0answers
59 views
Latent Dirichlet Allocation in R, topicmodels using VEM algorithm or Gibbs Sampling mixing tm and topicmodels library or WarpLDA from text2vec?
If I am trying to classify 230k text abstracts, which option would be better and more precise when aplying LDA?
5
votes
0answers
354 views
Gensim LDA model: return keywords based on relevance (λ - lambda) value
I am using gensim library for topic modeling, more specifically LDA. I have created my corpus, my dictionary and my lda model, and with the help of pyLDAvis library I visualize the results. When I ...
1
vote
1answer
41 views
how to do topic modeling on very huge data?
When i come to know that gensim is useful library for topic modeling, I tried it on my huge amount of document. It works well only if the dictionary size is to be fix. In my case i have each and every ...
0
votes
1answer
34 views
Which are the appropriate prameters for lda modeling?
I try to implement in R test for appropriate metrics for lda.
Here the way I try to use LDA
...
1
vote
1answer
44 views
Representation options of strings (keywords/topics) in models
What are all the possible ways to represent keywords in a machine learning model?
The two I am aware of are:
one hot encoding, using a static index.
vector representation, using
an embedding layer.
...
1
vote
0answers
19 views
Determining topic of text
I was wondering what I should be looking into if I want to measure the similarity between a paragraph and a corpus of text.
For example, given a paragraph of text and the entire corpus of Data ...
0
votes
0answers
35 views
Topic Modelling
New to python - topic modelling, trying to include bigrams in preprocessing
Had done the following,
...
1
vote
1answer
196 views
Find specific topics with topic modelling
I am looking for a way to classifiy text automatically by specific topics, i don´t have labeled data. Is this a possible/usual method of achieving this? If not, what would be better?
Topic Modelling ...
1
vote
1answer
38 views
Which approach to select category based on keywords
I want to assign a certain category to a group of keywords.
So i.e. people can upload images or videos, when they do this they can set keywords for this.
These keywords are free to type so words can ...
7
votes
4answers
4k views
BERT: it is possible to use it for topic modeling?
I'm struggling to understand which are the full capabilities of BERT: it is possible to make topic modeling of text, like the one we can achieve with LDA?
0
votes
0answers
50 views
Linking LDA topics to the input documents
I am new to LDA topic modelling. I am using gensim and am able to generate topics that make sense. Using 25k of documents, I can also print them using print_topics. ...
1
vote
1answer
41 views
Classify documents using a set of known vocabularies
I have a bunch of documents that I want to classify which ones talk about soccer (unsupervised learning, I do not want to manually label the documents).
One way I am thinking about is to go online ...
1
vote
0answers
13 views
How to find out the subject of an email (in the form of a sentence) or a pdf document in NLP using Python
How to find out the subject of an email (in the form of a sentence) or a pdf document in NLP using Python. If I do topic modelling and get different groups of topic, how do I pick out the only topic ...
2
votes
1answer
736 views
Online vs Batch Learning in Latent Dirichlet Allocation using Scikit Learn
Reference
I'm looking at the LDA algorithm from Scikit Learn for topic modeling. Can someone tell me how the 'online' method of learning works vs the 'batch' method of learning?
Also, what is learning ...
5
votes
1answer
1k views
NLP algorithms for categorizing a list of words with specific topics
Currently I am using LDA to apply topic modeling to a corpus. Since LDA is unsupervised, it returns a set of words for a given 'topic' but doesn't necessarily specify the topic itself. I was wondering ...
0
votes
1answer
365 views
Is it correct to create topic models using both train and test data?
I have a dataset of text documents splitted into train and test sets. My task is a binary classification, classifying these documents to either 1 or -1. I have already computed some features using TF-...
0
votes
1answer
60 views
Topic Segmentation - should it be done in Raw, TfIdf or Semantic Space?
Let's assume we have a collection of documents and wish to perform some unsupervised topic segmentation.
As always, we will perform some preprocessing (including tokenization, accent-removal, ...
2
votes
1answer
1k views
How to give names/labels to topics in LDA [duplicate]
I want to give labels to different topics created using LDA. I don't want to do it manually. I saw some papers on automatic labeling but I am still confused.
How can I use the information produced ...
2
votes
2answers
351 views
What's beyond topic modeling?
I tried topic modeling (LDA, NMF) to extract insights from the data.
I'm curious right now, are there other methods for unsupervised learning to cluster documents by the same or similar context?
(...
8
votes
1answer
19k views
Resume Parsing - extracting skills from resume using Machine Learning
I am trying to extract a skill set of an employee from his/her resume. I have resumes stored as plain text in Database. I do not have predefined skills in this case. How should I approach this problem?...
3
votes
1answer
66 views
Evaluating the result of topic modeling in a way that time matters
I have run different topic modeling approach on my data(its clinical data related to Cognitive impairment diseases. we are going to process what thing is important that make it develop to more harsh ...
1
vote
1answer
41 views
How to build News Tagging model(s)
I am trying to build a news tagging system. Given a piece of news article, find 5-6 key terms from the news article that best describe the article. Refer to the image below from google news.
What are ...
1
vote
1answer
631 views
How to compare the topic coherence between models of different number of topics?
If I'm not mistaken, in this paper here http://svn.aksw.org/papers/2015/WSDM_Topic_Evaluation/public.pdf it appears that topics with larger number of topics will inherently have larger coherence ...
6
votes
1answer
286 views
How to split natural language script into segments?
I have a bunch of .txt and .srt files extracted from a MOOC website, they are the scripts of the videos. I would like to segment the scripts into parts such that each part falls into one of the ...
4
votes
1answer
429 views
Why do we need the hyperparameters beta and alpha in LDA?
I'm trying to understand the technical part of Latent Dirichlet Allocation (LDA), but I have a few questions on my mind:
First: Why do we need to add alpha and gamma every time we sample the equation ...
0
votes
2answers
304 views
Guided topic modeling: generating words from topics
I need to generate lists of words related to specific topics for a project. I am familiar with clustering methods of topic modeling such as LDA, but I have something else in mind. Are there any ...
1
vote
0answers
101 views
Performance Metric for topic extraction when there is no ground truth
I am extracting topics from text using a predefined ontology containing 2690 concepts, wordnet(to expand concept terms with their synsets, and other morphological forms of the same word) and lucene to ...
3
votes
2answers
107 views
Collection Of Variable Length Sequences and Descriptions: A Search Problem
I have a tough problem and need some advice:
Suppose I have a collection of variable length sequences, many of which are unique -- imagine the moves to a chess game, eg
d4 Nf6
c4 g6
Nc3 Bg7
...
9
votes
1answer
4k views
What is the difference between topic modeling and clustering?
I know that topic modeling and clustering are related, but not similar techniques. Can anyone suggest what are the main differences?
