I have a binary classification task which has the following specification:
Input: Chunk of text (not more than a few sentences, mostly a sentence).
Additional Input: For each input sample there is additional information available (which is also some text of similar length, max 2-3 sentences).
Problem: Classify text content using additional context
Problem Type: Binary classification
Essentially the task boils down to classifying the content conditioned on the context or p(content|context).
I was thinking of effective ways to encode the text for classification using a deep neural network. I searched for recent works but the existing literature mostly use the technique mentioned below.
I was wondering if there is a better way to encode context? Which has shown to be effective in some domain as compared to just concatenation.
Encode then concatenate: This involves encoding the context using ways similar to the content and then concatenating the feature representation before classification. This is the widely used technique. There are a lot of variants of this technique (different ways to encode text e.g. using tfidf rep, word embedding, LSTMs, CNNs ) which are widely explored.
Are there any better ways to bring context information during classification?
P.S: Recursive RNN is something that I have on my TODO.