0
$\begingroup$

I am using top2vec to perform topic modelling.

According to the paper, topic reduction can be performed on the topic vectors to hierarchically group similar topics and reduce the number of topics discovered.

This would be very useful because the topics found by the algorithm on my dataset are pretty similar to each other.

Does anyone know if there is some python code to do this? On the online documentation, there is nothing about it.

I have also opened an issue on the official page if someone wants to post an answer there.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

I like the initiative that you answered your own question.
In case you would like an additional answer: What you are referring to in your question is the concept of hierarchical clustering, a great way to cluster since it is deterministic (meaning no matter how many times you run it, the groupings are the exact same, unlike k-means clustering) and the hierarchical nature of the clustering means you can choose the number of higher-level topics via a overall intra-cluster distance threshold. A clear example of how to implement it can be found here: Hierarchical Clustering in Python

Good luck!

Improve this answer
$\endgroup$
1
  • $\begingroup$ Hi, I have understood after a quick read of Angelov's paper that the Top2Vec algorithm already executes an HDBSCAN clustering. So I think there should be a way to do it without needing to perform an additional clustering algorithm. $\endgroup$
    – Alberto De Benedittis
    Feb 8 at 16:46
1
$\begingroup$

I do not know if this is the smartest way to do it, but it seems to work for my study case. To recap, I performed a top2vec algorithm:

import numpy as np 
import pandas as pd 
from top2vec import Top2Vec

df = pd.read_excel('Preprocessed2.xlsx')
documents = df_small_small['Translated Cleaned'].to_list()
model = Top2Vec(documents, embedding_model='universal-sentence-encoder', speed='deep-learn')

The algorithm finds 30 topics. However, although coherent, they could be grouped into smaller macrotopic since there are some similarities among different topics.

After having inspected them thanks to a word cloud visualization

topic_sizes, topic_nums = model.get_topic_sizes()
for topic in topic_nums:
    model.generate_topic_wordcloud(topic)

and printing closer topics to a specific keyword

topic_words, word_scores, topic_scores, topic_nums = model.search_topics(keywords=["queue"], num_topics=5)

topic_nums

>>> array([ 5,  7, 15,  1,  3], dtype=int64)

I was able to detect which topics could have been grouped together and I manually created a list with the indices of the topic that refer to the same macrotopic.

# for example 
macrotopic_1 = [1,7,12,17]

Then I implemented this two dumb functions to get the best words and documents for that best represented the macrotopic

def macrotopic_words(topics_words:np.ndarray, word_scores:np.ndarray, list_of_topics:list):
    ''' 
    This function takes in input the output of the top2vec model and a list of integers corresponding to topics'ids  and returns two dictionaries containing the most important words and their respective value 
    
    Input:

    - topic_words: (np.ndarray), an array of arrays containing topic words.
    - word_scores: (np.ndarray), an array of arrays containing words scores. These scores represent the cosine similarity to each keyword that defines the topic.  
    - list_of_topics: (list), a list of integers. These integers have to be between 0 and the max number of topics spotted by the top2vec algorithm 
    
    Output: 

    _ diz_words: (dict) dictionary whose keys are the top 50 words and values are their associated score. These scores represent the cosine similarity to each keyword that defines the topic. 
    '''
    # The two objects that will be returned
    diz_words = dict()
    
    for topic_id in list_of_topics: 
        topic_words = topics_words[topic_id].tolist()
        topic_words_scores = word_scores[topic_id].tolist()
        for i in range(len(topic_words)): 
            word = topic_words[i]
            score = topic_words_scores[i]
            if word in diz_words:
                old_score = diz_words[word]
                diz_words[word] = (old_score + score)/2
            else:
                diz_words[word] = score 
    
    # get only the top 50 words
    diz_words = dict(sorted(diz_words.items(), key=lambda item: item[1], reverse=True)[:50])

    return diz_words


def macrotopic_docs(df, model, list_of_topics:list):
    ''' 
    This function takes in input the output of the top2vec model and a list of integers corresponding to topics'ids  and returns two dictionaries containing the most important words and their respective value 
    
    Input:
    - df: (pd.Dataframe) datframe where I there are at least two columns: one with the orginal text (in my case translated reviews) and one with the pre-processed text (in my case translated reviews). 
          This is not necessary but for my application makes the result more human readable. 
    - model: (top2vec.Top2Vec.Top2Vec) output of the top2vec model
    - list_of_topics: (list), a list of integers. These integers have to be between 0 and the max number of topics spotted by the top2vec algorithm 
    
    Output: 

    _ diz_docs: (dict) dictionary whose keys are the top 50 words and values are their associated score. These scores represent the cosine similarity to each keyword that defines the topic. 
    '''

    # The two objects that will be returned
    diz_words = dict()
    
    for topic_id in list_of_topics: 
        documents, document_scores, document_ids = model.search_documents_by_topic(topic_num=topic_id, num_docs=10)
        for i in range(len(documents)): 
            doc = documents[i]
            doc_nice_idx = df.index[df['Translated Cleaned'] == doc][0]
            doc_nice = df.iloc[doc_nice_idx]['Translated reviews']
            score = document_scores[i]
            if doc_nice in diz_words:
                old_score = diz_words[doc_nice]
                diz_words[doc_nice] = (old_score + score)/2
            else:
                diz_words[doc_nice] = score 
    
    # get only the top 50 words
    diz_words = dict(sorted(diz_words.items(), key=lambda item: item[1], reverse=True)[:10])

    return diz_words

macrotopic_words(topic_words,word_scores,[5,7,8] )
>>> {'ticket': 0.3825201094150543,
 'queue': 0.2912799119949341,
 'long': 0.2060506008565426,
 'close': 0.2041976973414421,
 'min': 0.19961485266685486,
 'hour': 0.18805542960762978,
 'stop': 0.1846560537815094,
 'near': 0.1766391098499298,
 'longer': 0.17402754724025726,
 'forget': 0.1692977249622345,
 'open': 0.16631683707237244,
 'interest': 0.16506552696228027,
 'quit': 0.16504628770053387,
 'finish': 0.16126474738121033,
 'tail': 0.16030896827578545,
 'end': 0.15837934613227844,
 'much': 0.15592673420906067,
 'start': 0.15537860989570618,
 'stay': 0.15466280281543732,
 'wait': 0.15337903797626495,
 'visit': 0.1528526358306408,
 'time': 0.15138189494609833,
 'work': 0.14888891577720642,
 'gate': 0.14842548966407776,
 'worth': 0.1457190401852131,
...
 'full': 0.13024736940860748,
 'second': 0.13011334836483002,
 'buy': 0.1292862892150879,
 'moment': 0.12763935327529907,
 'opportun': 0.1272534765303135}

macrotopic_docs(df, model, [1])
>>> {'On October 3, the Pergamon Museum reopened its gates.The pergamon altar is not yet available.But even that\'s how you can spend interesting hours in the old Orient.Highlights such as the Ischtar goal, the Milet or Diemschatta facade, are accessible.Very good audio guide with pleasantly compact explanations.It was not so nice that you are currently running "backwards" through the exhibition and the audio guide is not adapted to this.This leads to the fact that you can hear the introduction to your specific exhibition area just before you leave this room ... In the introduction, the audio guide also indicates that the pergamon altar is expected to be restored until 2019!In the months of the closure, one could really have bothered to update the audio guide! ...': 0.7808947,
 "The Pergamon Museum is an obligatory stop in Berlin.Ishtar's door alone is worth the ticket, but don't stop there.The whole exhibition is absolutely enviable, especially if you take the free audio guide at the entrance, which tells you anecdotes and the story of Pergamo. ...": 0.7628059,
 'Beautiful experience. Babilonia Porta magnificent as well as the entry to the Miletus market. Make up the audio guides included in the price of our ticket (Euro 19 on Get Your Guide, with a priority entrance), which has a very good value for moneywhich also provides ...': 0.75661397,
 'If anyone goes to Berlin, he must visit this place.Huge impressions !!!Personally, I think that you should ensure a lot of time to visit.The museum delights !!!In addition, at the entrance we get an audio guide and we decide how much time we want to spend on ...': 0.74968064,
 "I've never seen anything like that, this museum is unforgettable!The audio guide helps a lot, it's worth it!And it was the first time I saw a hygiene being done in audio guides.They clean each one and give you a cover to the headphones.wonderful!Hot Tip: Buying online is cheaper and cut…": 0.74947065,
 "I have already visited this museum twice;It is wonderful, there are entire parts of temples, a piece of the Babylon gate, it is really great.The audio guides allow a guided tour according to the visitor's time and interests.": 0.7454577,
 'Unmissable museum if passing through Berlin.Unfortunately, the Pergamo section was not open, but the Babylon Palace and the Mileto market door are certainly worth the trip.Excellent audio guides provided free of charge with the ticket.It is preferable to do the ...': 0.74440473,
 "The pergame is wonderful.Everyone who visits Berlin must pass by.The entrance costs € 12 and if you don't have much time, in two hours you can travel it.Free guide and wardrobe audio.Keep in mind that a part of the museum is closed and its great star, the ...": 0.7332305,
 'I saw it very willingly because it really represents a piece of history in a unique way.Both the Temple of Pergamon, and the Babylon gate manage to leave you without a word.The audio guide, included in the price of the entrance ticket, is very well done and accompanies ...': 0.7329705,
 'I visited Berlin with the family and made a ticket that allowed us to enter all the museums.We made a rather quick visit to this museum but the emotion in seeing the area dedicated to the Ishtar gate was great.Very beautiful the audio guide who ...': 0.7325742}
```
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.