Assuming you are doing supervized learning to train a model that when deployed will take text as input and output a label (e.g., topic) or class porbability, then what you probably want to do is balanced, [stratified sampling][1]. Assuming sufficient labelled data, ensure that your final training set has a balanced number of text examples for each class/label. Depending on your situation, you may need to over/under sample or somehow deal with the problem of highly imbalanced classes (see [8 tactics to combat imbalanced classes][2]). The simplest NLP approach is to use a [bag of words][3] technique, simply indicating the presence/absence of a word in the sentence. Thus each sentence becomes represented as vector of length n, where n = the number of unique words in your data set. Sometimes you can increase performance by adding [bi-gram][4] and tri-gram features. Sometimes weighting words by their frequency improves performance or computing the [tf-idf][5]. Another to increase performance in my experience, has been custom language feature engineering, especially if the data is from social media replete spelling errors, acronyms, slang, and other word variants. [1]: https://en.wikipedia.org/wiki/Stratified_sampling [2]: http://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/ [3]: https://en.wikipedia.org/wiki/Bag-of-words_model [4]: https://en.wikipedia.org/wiki/Bigram [5]: http://blog.christianperone.com/2011/09/machine-learning-text-feature-extraction-tf-idf-part-i/