2
$\begingroup$

I am trying to classify multi-page documents using a convolutional neural network (CNN). The content of each page in the corpus contains only text (i.e., no photographs or icons), and different documents may have a different shape (height and width). I'd like my classification approach to use all pages in each document, rather than just the first page.

As far as I know, the input to a CNN (like theano) needs a standardized shape. My first thought was to create a single image array that has all the pages concatenated. But I would then have to resize/zero pad all concatenated pages to match the length of the document with the largest page count, and use that as my lowest height and width for the set of concatenated pages. If I don't use this strategy I risk losing resolution on the words in the image, but that is a giant input vector.

I feel like splitting each document's pages into separate samples would be a better approach to standardize input preprocessing, but I'm at a loss for how to classify the whole document if I'm just training on loose pages of each document. Can anyone advise on me on this?

$\endgroup$
  • $\begingroup$ You could sum and then normalize on page count - square root of the sum of the squares - harmonic mean - average - product. $\endgroup$ – paparazzo Dec 11 '15 at 17:43
  • $\begingroup$ What are you classifying? If you are classifying what the document "looks like" then using an image would make sense. But if you are classifying what the text actually says, then using an RNN built using the text or a bag of words model may be better suited. $\endgroup$ – user1269942 Dec 13 '15 at 5:45
  • $\begingroup$ I don't have OCR text of the documents. But yes I'm trying to achieve the classification of what the documents contents "actually says", without the OCR text corpus as input. My thought is the X would be the image pixels and the Y would be the type of document it is (example: scify, romance novel, etc..). $\endgroup$ – jams Dec 13 '15 at 18:23
1
$\begingroup$

First point is that convolutional neural networks would be incredibly expensive to train on images that large. The minimum size you could allow would be num_pixels_per_letter x num_letters_per_row_or_col.

It sounds like you want to achieve two tasks: OCR then document classification. Given that you have predominantly text in these documents, you would almost certainly be better served by using a more a traditional method for OCR. If you want or need to use NNs for the OCR piece, you will almost certainly need to use a sliding window. For inspiration on how you might do that, you could look to Recurrent Neural Networks that process images in sequences

Once you have the text there are again much simpler methods to deal with text than NNs, but they can achieve great results. To deal with different document lengths, you can use an RNN architecture like LSTM. Or you can use paragraph vectors, which do most of the heavy lifting for you and give you an N-dimensional representation of your text.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.