If you are asked to do text categorization using clustering. Which algorithm would you use and why?
It will depend on the purpose and the text. Many options this is what I have used.
What I did was limit the cluster and any document vector to the top 1000 terms sorted by weight. This not only results in faster processing but you get some multi modal clusters and will have thousands and thousands of terms. What happens is those really long vectors dilute the similarity when compared to a even a long single document. I think you also get faster convergence. If you want to keep documents out of multi modal clusters then don't truncate the cluster vector.
tf-idf vectors are an easy start, but it can be hard to cluster very high dimensional data.
You might try topic modeling (LDA, LSI for example) to reduce the dimensionality of your features.
Learning a reasonable, lower dimensional representation of the text can help with the issues that arise trying to cluster high dimensional data.