Topic Segmentation - should it be done in Raw, TfIdf or Semantic Space?

Question

Let's assume we have a collection of documents and wish to perform some unsupervised topic segmentation.

As always, we will perform some preprocessing (including tokenization, accent-removal, lowercasing, lemmatizing and such) and transform the lists of tokens into either raw-counts or tfidf-vectors. We'll call this matrix M.

Now we have several possible approaches to perform some simple bag-of-words topic segmentation:

• Apply a matrix decomposition method (LSI, LDA, NMF) directly to M and use the resulting components as the topics.
• Embed each vector of M into a semantic space (LSI, word2vec) and then apply a matrix decomposition method on the semantic space.
• Apply a clustering method (kM, DBSCAN, MSC, GMM) directly to M.
• Embed each vector of M into a semantic space and then apply a clustering method on the semantic space.

I have two questions:

1. Are there any other alternatives to bag-of-words topic segmentation that I have not considered yet?
2. What are the conceptual differences between the methods described above and which one(s) are recommended?

Thanks in advance!

Answer 1

1. Clustering method can applied directly to TfIdf matrix (which will be generally sparse) or to documents in the derived semantic space like LSI. Since LSI embedding is a dimensionality/noise reduction step, it is a good idea to cluster on documents embedded in semantic space. Clustering methods will generally use Euclidian distance and will probably perform better on dense matrix. If the clustering algorithm implementation that you use doesn't support sparse data out of the box (most implementations don't) then definitely use a transformation like LSI first and then apply clustering.

2. LSI method itself is a matrix decomposition operation on the original TfIdf matrix. So applying matrix decomposition on top of LSI is not needed.