Skip to main content

Topic Modeling would be a very appropriate method for your problem. Topic Models are a form of unsupervised learning/discovery, where a specified (or discovered) number of topics are defined by a list of words that have a high probability of appearing together. In a separate step, you can label each topic using subject matter experts, but for your purposes this isn't necessary since you are only interested in getting to three clusters.

You treat each document as a bag of words, and pre-process to remove stop words, etc. With the simplest methods, you pre-specify the number of topics. In your case, you could either specify "3", which is your fixed limit on categories, or pick a larger number of topics (between 10 and 100), and then in a separate step, form three clusters for documents with common emphasis on topics. K-means or other clustering methods could be used. (I'd recommend the latter approach)

You don't need to code topic modeling software from scratch. Here's a web page with many resources, including software libraries/packages: http://www.cs.princeton.edu/~blei/topicmodeling.htmlweb page with many resources, including software libraries/packages.

None are in Java, but there are ways to run C++ and Python under Java.

Topic Modeling would be a very appropriate method for your problem. Topic Models are a form of unsupervised learning/discovery, where a specified (or discovered) number of topics are defined by a list of words that have a high probability of appearing together. In a separate step, you can label each topic using subject matter experts, but for your purposes this isn't necessary since you are only interested in getting to three clusters.

You treat each document as a bag of words, and pre-process to remove stop words, etc. With the simplest methods, you pre-specify the number of topics. In your case, you could either specify "3", which is your fixed limit on categories, or pick a larger number of topics (between 10 and 100), and then in a separate step, form three clusters for documents with common emphasis on topics. K-means or other clustering methods could be used. (I'd recommend the latter approach)

You don't need to code topic modeling software from scratch. Here's a web page with many resources, including software libraries/packages: http://www.cs.princeton.edu/~blei/topicmodeling.html

None are in Java, but there are ways to run C++ and Python under Java.

Topic Modeling would be a very appropriate method for your problem. Topic Models are a form of unsupervised learning/discovery, where a specified (or discovered) number of topics are defined by a list of words that have a high probability of appearing together. In a separate step, you can label each topic using subject matter experts, but for your purposes this isn't necessary since you are only interested in getting to three clusters.

You treat each document as a bag of words, and pre-process to remove stop words, etc. With the simplest methods, you pre-specify the number of topics. In your case, you could either specify "3", which is your fixed limit on categories, or pick a larger number of topics (between 10 and 100), and then in a separate step, form three clusters for documents with common emphasis on topics. K-means or other clustering methods could be used. (I'd recommend the latter approach)

You don't need to code topic modeling software from scratch. Here's a web page with many resources, including software libraries/packages.

None are in Java, but there are ways to run C++ and Python under Java.

Source Link
MrMeritology
  • 1.8k
  • 13
  • 14

Topic Modeling would be a very appropriate method for your problem. Topic Models are a form of unsupervised learning/discovery, where a specified (or discovered) number of topics are defined by a list of words that have a high probability of appearing together. In a separate step, you can label each topic using subject matter experts, but for your purposes this isn't necessary since you are only interested in getting to three clusters.

You treat each document as a bag of words, and pre-process to remove stop words, etc. With the simplest methods, you pre-specify the number of topics. In your case, you could either specify "3", which is your fixed limit on categories, or pick a larger number of topics (between 10 and 100), and then in a separate step, form three clusters for documents with common emphasis on topics. K-means or other clustering methods could be used. (I'd recommend the latter approach)

You don't need to code topic modeling software from scratch. Here's a web page with many resources, including software libraries/packages: http://www.cs.princeton.edu/~blei/topicmodeling.html

None are in Java, but there are ways to run C++ and Python under Java.