1
$\begingroup$

Please advise on starting points, research (papers,frameworks) related to thematic clustering of text. In particular on a system with two levels of clustering where second level has a temporal nature. Thanks!

Update:

Sorry for ambiguity in my initial question. I need to clarify, that I have experience with clustering in general and document clustering in particular, for text using TFIDF, word embeddings (word2vec, Glove and BERT Sentence embeddings) in a vector space.

My question originates from some text mentioning "two levels of thematic clustering of text where second level has a temporal nature". Just wanted to know about this technique and in particular "temporal nature" used in clustering.

$\endgroup$
1
$\begingroup$

Your question is way to vague. What did you try, what do you want to achieve in detail. However, find below some R examples how to approach topic modeling. This book might also be helpful: Text Mining with R.

library(topicmodels)
data("AssociatedPress")
AssociatedPress

# LDA (Latent Dirichlet Allocation)
ap_lda <- LDA(AssociatedPress, k = 2, control = list(seed = 1234))
ap_lda

# Get "topics"
library(tidytext)
ap_topics <- tidy(ap_lda, matrix = "beta")
ap_topics

library(ggplot2)
library(dplyr)

# Plot topics
ap_top_terms <- ap_topics %>%
  group_by(topic) %>%
  top_n(10, beta) %>%
  ungroup() %>%
  arrange(topic, -beta)

ap_top_terms %>%
  mutate(term = reorder(term, beta)) %>%
  ggplot(aes(term, beta, fill = factor(topic))) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~ topic, scales = "free") +
  coord_flip()

# (words with) Greatest difference between categories
library(tidyr)

beta_spread <- ap_topics %>%
  mutate(topic = paste0("topic", topic)) %>%
  spread(topic, beta) %>%
  filter(topic1 > .001 | topic2 > .001) %>%
  mutate(log_ratio = log2(topic2 / topic1))

beta_spread

list(beta_spread$term)

# Select 10 largest and smallest values of "log_ratio")
df1 <- subset(beta_spread, beta_spread$log_ratio<=max(tail(sort(beta_spread$log_ratio, decreasing = T),10)))
df2 <- subset(beta_spread, beta_spread$log_ratio>=min(tail(sort(beta_spread$log_ratio, decreasing = F),10)))
df = rbind(df1,df2)
df

# Plot
barplot(sort(df$log_ratio), names.arg=df$term[order(df$log_ratio)], las=2)

# Gamma contains the probability of document X to belong to topic Y
ap_documents <- tidy(ap_lda, matrix = "gamma")
ap_documents
# the model estimates that only about 24.8% of the words in document 1 were generated from topic 1
$\endgroup$
2
  • $\begingroup$ Thanks! Is thematic clustering the same thing as topic modeling? Where does temporal nature comes into play? $\endgroup$ – dokondr Apr 5 '20 at 20:38
  • $\begingroup$ Please, see my question update. $\endgroup$ – dokondr Apr 6 '20 at 9:00
1
$\begingroup$

"Clustering" is a very broad umbrella for a set of unsupervised techniques that tries to group data items together, according to its characteristics/covariates/variables.

Let's say you have two variables $x1$ and $x2$, and 6 samples. I could easily find two clusters with techinques such as k-Means.

enter image description here

When it comes to text mining, these variables are frequently associated with word frequency and/or context representation. For example, a sample may be a document and $x1$ and $x2$ may be the frequency of word "a" and word "b". Also, timestamp could be $x3$.

If you want to find clusters among documents, you need first to define a variable or "feature" extraction method (such as word frequency, tf-idf, word embedding etc). You can concatenate your text features with time-related features, and apply any clustering technique to this set of features in order to cluster your documents.

@Peter suggest you to use a topic modelling technique, which is a method for reducing the feature dimensional space (2 features = 2 dimensions, 1000 features = 1000 dimensions) after applying a word frequency feature extraction. It will help you describe each of these documents according to the frequency of a certain set of important words. Roughly speaking, a topic is a set of words that appear together. So for each document, the topics will have a relevance level.

It is not strictly a "clustering" approach, but it certainly achieves document clusterization by using the most relevant topic.

If you want to have temporal dimension coupled with topic modelling, you have to study a little more, read some papers and have some practice with these methods I mentioned. It's possible to have a pipeline with topic modelling for the word frequency ($x1$ is the relevance of topic 1 for a document, $x2$ for topic 2 ...) then you attach the timestamp to the result and apply a clustering technique such as k-Means.

$\endgroup$
2
  • $\begingroup$ Please, see my question update. $\endgroup$ – dokondr Apr 6 '20 at 9:01
  • $\begingroup$ You need to think that "temporal nature" is another set of variables. You can attach "temporal" variables to your feature set and perform the cluster normally. Think about what if you have other document information (such as the age of the writer), it is reasonable to say that you will need a different approach for clustering itself just because you have this new information? No, the clustering process is the same, you only have a few more variables that you need to extract from the timestamp. $\endgroup$ – Adelson Araújo Apr 6 '20 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.