Since min_count and threshold are hyperparameters, better values could be found through cross validation. Evaluate a range of values to empirically find the values that have the highest performance on a validation set.
It depends on the type of ranking that you want to achieve, for example if the unlabeled scraped data can be ranked by sentiment, you can use Transfer Learning models to give each document a sentiment score which will serve as a rank if you return the sentiment score probability instead of having "positive" and "negative" tags.
Running this now, and updating to import GridSearchCV from model_selection, the code is:
from sklearn.datasets import fetch_20newsgroups
# from sklearn.grid_search import GridSearchCV
from sklearn.model_selection import GridSearchCV
categories = ['sci.med', 'soc.religion.christian']
First I think it's worth mentioning that in the context of an exploratory study with a small dataset, manual analysis is certainly as useful as applying NLP methods (if not more) since:
Small size is an advantage for manual study and a disadvantage for automatic methods.
There's no particular goal other than uncovering general patterns or insights, so it's ...