Word2Vec with CNN

Question

I am trying to classify documents using CNN (convolutional neural network) with Word2Vec embeddings. However to do this, it requires me to trim all texts to the same length. I just pad all the training documents to the size of the longest, and I don't think this is the best solutions, as during the testing phase, there can come a longer document and I may remove a significant part of it by trimming.

I found that there is Doc2Vec, which may solve this problem, but I don't understand how it can be used in conjunction with CNN.

Answer 1

I think RNNs (LSTM, GRU etc.) is more suitable for text classification.

CNNs has a fixed window size, thus you will always need to pad the input into same length.

I took a cursory look at the Doc2Vec document. It says

Doc2Vec is a Model that represents each Document as a Vector.

Given only one vector for each document, it's meaningless to use CNNs as a classifier. Since CNNs is used to capture the local pattern or something.

However, you can split the document into sentences or paragraphs, but you will still need to pad it into same length.

Answer 2

The idea is that Doc2Vec is an average of all your word vectors. Your document embedding will be the same size as any other word vector you have. If you average 10 word embeddings of size 100x1 or if you average 20 word embeddings of size 100x1 you will in both cases get a document embedding of 100x1. So then, the size of the sentence doesn't matter and no padding is applied.

You can then take that 100x1 vector and pass it through a CNN like you would for any other type of input into a CNN.