0
$\begingroup$

I have training data in form of pair of documents with an associated label - {doc1, doc2, label}. Label is defined as function of pair of documents.

Now I want to build a model which can predict the label given two new documents.

I want to try different representation of document (instead of common ones say TF-IDF). Can I use vectors (topic distribution) from LDA as features for a classifier?

$\endgroup$
0
$\begingroup$

Yes, that is a reasonable approach. Also try neural network based representations such as doc2vec. I suppose you know how to do the classification part?

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer. Yeah, I can try other representation schemes also, like doc2vec, avg. of word vectors etc. And for classification part I am planning to use a simple Logistic regression model on top of concatenation of the two vectors (one from each document). Does this sounds reasonable? $\endgroup$ – SHASHANK GUPTA Sep 11 '16 at 21:01
  • $\begingroup$ Instead of concatenation, I'd lean towards learning an inner product; have the input to sigmoid function be $x_1^\dagger M x_2$, for a PSD M, where the parameter to be learned is now the matrix $M$. But go ahead try it your way, see what you get. $\endgroup$ – Emre Sep 11 '16 at 21:19
  • $\begingroup$ Yeah, I saw this being used as one feature in a paper. But they also used concatenation of vectors. I mean th feature vector was [x1;x2;x1.T.M.x2]. Why are you suggesting use of this as feature instead of concatenation or as suggested in that paper? I mean is there any advantage of this one over that? I am guessing this feature will learn an similarity function (characterised by matrix M) between x1 and x2? I am not sure though. $\endgroup$ – SHASHANK GUPTA Sep 11 '16 at 21:27
  • $\begingroup$ That sounds good too. $\endgroup$ – Emre Sep 11 '16 at 21:31
  • $\begingroup$ Yeah, I'll try that. Thanks for your help :). Just one question, am I right in my understanding that this feature learns a similarity metric (M) between two vectors? $\endgroup$ – SHASHANK GUPTA Sep 11 '16 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.