1
$\begingroup$

I build a classifier of documents using the vector representation of each document in the training set (i.e a row in the Document-Term Matrix). Now I need to test the model on the test data. But how can I represent a new document with the Document-Term Matrix since some terms might not be included in training data?

$\endgroup$
0
$\begingroup$

The easiest way is to treat all out-of-vocabulary terms as a specific term in your matrix (i.e. "OOV").

So for instance, if my training data contains 3 words: "I", "like", "cake", my document-term matrix would contain 4 items, "I", "like", "cake", and "OOV".

| improve this answer | |
$\endgroup$
  • $\begingroup$ Is there a way to automate this procedure? $\endgroup$ – Paw in Data May 5 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.