I'm trying to use CNN (convolutional neural network) to classify documents. CNN for short text/sentences has been studied in many papers. However, it seems that no papers have used CNN for long text or document.
My problem is that there are too many features from a document. In my dataset, each document has more than 1000 tokens/words. To feed each example to a CNN, I convert each document into a matrix by using word2vec or glove resulting a big matrix. For each matrix, the height is the length of the document, and the width is the size of word embedding vector. My dataset has more than 9000 examples and it takes a lot of time to train the network (a whole week) which makes it difficult to fine-tune parameters.
Another feature extracting method is to use one-hot vector for each word, but this will create very sparse matrices. And of course, this method even takes more time to train than the previous method.
So is there a better method for extracting features without creating large input matrices?
And how should we handle variable length of documents? Currently, I add special strings to make document have the same length, but I don't think it's a good solution.