0
$\begingroup$

I am working on a dataframe containing abstracts from various NLP conferences, along with information on information on the respective authors (names) and the keywords they've associated with their abstracts.

e.g.

  • abstract
  • author1, author2
  • kw1, kw2, kw3

My objective is to cluster authors who frequently write about similar topics, as indicated by shared keywords. For the visualisation I am thinking of using t-SNE. However, I am unsure about specifying 'cluster labels' without manual intervention. Which algorithms would be suitable for this task* ?

*e.g. would K-means be a viable option given that number of clusters should be provided in advance? or should I opt for methods such as DBSCAN or Affinity Propagation ? Should I consider keywords as clusters (risking of producing an explosion of clusters because of the large number of keywords)

$\endgroup$
0

1 Answer 1

1
$\begingroup$

My baseline for this task is as follows (to analyze the abstract):

  • tokenize abstract,
  • normalize tokens,
  • vectorize tokens (FastText, BERT, ...),
  • add vectors,
  • cluster with DBSCAN, changing threshold and analyzing clusters. It's fashionable to try to do the same with keywords.

But it's all baseline. Advanced topic modeling of scientific articles requires models that take into account the hierarchy of topics, different modalities (abstract s and keywords), and so on. For a similar task, I recently used BigARTM.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.