I've been working on a NN-based classification system that accepts document vectors as input. I can't really talk about what I'm specifically training the neural net on, so i'm hoping for a more general answer.
Up to now, the word vectors I've been using (specifically, the gloVe function from the text2vec package for R) have been target vectors. Up to now I wasn't aware that the word2vec training produced context vectors, and quite frankly I'm not sure what exactly they represent. (It's not part of the main question, but if anybody could point me to resources on what context vectors are for and what they do, that would be greatly appreciated)
My question is, how useful are these context word vectors in any kind of classification scheme? Am I missing out on useful information to feed into the neuralnet?
How would, qualitatively speaking, these four schemes fare?
- Target word vectors only.
- Context word vectors only.
- Averaged target and context vectors.
- Concatenated vectors (i.e. a 100-vector word2vec model ends up with a length of 200)