0
$\begingroup$

I am working on a multi-label document classification task with a very small data set (180 labeled documents) and a fairly large number of labels (20).

I found that - ignoring label correlations and turning the problem into 20 binary decision problems - tf-idf features fed into a logistic regression work reasonable well.

Since I have domain knowledge I would like to add manual weighs to some features to increase the prediction accuracy. For instance, assume I train a model to recognise the category "Gender", I am thinking about something like this:

'''extract features using tfidf vecorization:'''

vectorizer = TfidfVectorizer(ngram_range = (1,2),min_df = 0.01, max_df = 0.95)
vect = vectorizer.fit(X_train)
X_train = vect.transform(X_train)
X_test = vect.transform(X_test)

'''add additional feature weight'''

weight = 10
position = vect.vocabulary_['woman']
X_train[:, position] *= weight
X_test[:, position] *= weight
position = vect.vocabulary_['gender']
X_train[:, position] *= weight
X_test[:, position] *= weight

The same weighting is then applied to the vectorised unseen data that I want to predict.

This indeed gives me a better accuracy on the test set and also leads to better predictions on the unseen data, however, I feel quite uncomfortable to "manually" change the data.

I have not found any information about such manual feature tuning. Is this something that can be justified?

Bonus question: Tf-idf seems to work the best to build features for longer documents (>2000 characters), however, do you have another suggestion - personal positive experience - to build a feature space for long documents?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.