I am working on a multi-label document classification task with a very small data set (180 labeled documents) and a fairly large number of labels (20).
I found that - ignoring label correlations and turning the problem into 20 binary decision problems - tf-idf features fed into a logistic regression work reasonable well.
Since I have domain knowledge I would like to add manual weighs to some features to increase the prediction accuracy. For instance, assume I train a model to recognise the category "Gender", I am thinking about something like this:
'''extract features using tfidf vecorization:''' vectorizer = TfidfVectorizer(ngram_range = (1,2),min_df = 0.01, max_df = 0.95) vect = vectorizer.fit(X_train) X_train = vect.transform(X_train) X_test = vect.transform(X_test) '''add additional feature weight''' weight = 10 position = vect.vocabulary_['woman'] X_train[:, position] *= weight X_test[:, position] *= weight position = vect.vocabulary_['gender'] X_train[:, position] *= weight X_test[:, position] *= weight
The same weighting is then applied to the vectorised unseen data that I want to predict.
This indeed gives me a better accuracy on the test set and also leads to better predictions on the unseen data, however, I feel quite uncomfortable to "manually" change the data.
I have not found any information about such manual feature tuning. Is this something that can be justified?
Bonus question: Tf-idf seems to work the best to build features for longer documents (>2000 characters), however, do you have another suggestion - personal positive experience - to build a feature space for long documents?