0
$\begingroup$

Lately, I have been largely inspired by this https://rossum.ai/, which is able to extract text from invoice documents.

Do you have any ideas on how this could be implemented? It's clear that they did a lot of research to reach this performance level, but in my case I am interested in the overall approach to such problems.

If I understand correctly, the first part of the pipeline is to extract different blocks from the document. In that case, is object detection the right approach to get bounding boxes around the blocks? I guess it might not be really good at extracting tabular data.

If not object detection, what is the correct way to tackle the problem?

Thanks.

$\endgroup$

2 Answers 2

0
$\begingroup$

I think extracting relevant details from an invoice in commercial applications certainly involves a lot of high spec algorithms. Maybe you are right that they identify relevant parts first and extract the details afterwards.

However, my first starting point would be to get all the text from an invoice (e.g. via tesseract). If you have a decent photo, tesseract will be able to OCR the content. The next step would be to identify relevant content, such as payment amount, names, and bank account numbers. This may be possible by hardcoded rules to some extent. Alternatively, one could use NLP-like models to detect certain sequences. With some effort, this should work out well since invoices are relatively structured documents.

https://pypi.org/project/pytesseract/

https://github.com/tesseract-ocr/tesseract/wiki

$\endgroup$
0
$\begingroup$

There are a couple of approaches you can use for this.

The first involves using regular expressions and some string manipulation to extract the relevant details. This can only work if the layout of the document is simple and the types of layout are not too diverse.

The second more robust and accurate approach is to use AI to extract the details. This approach involves 2 components. One is the OCR Engine and the second is a Deep Learning model. First you have to extract all the text present in the document using the OCR Engine. After extracting the text, feed the text to a Deep Learning model (you'll have to train the model on your custom data first!), so that the model can extract the required details.

I have delivered multiple similar projects and in my opinion the best approach is to combine both the approaches (kind of a hybrid approach) to get the maximum accuracy.

Cheers!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.