I am trying to find specific information from supplier invoices using ML. Now, I have a lot of invoices I can use for test (+100k). I figure the first step is to label them, so I can train a model to extract the information I need.
Below is a sample invoice (I have anonymized it by removing the shipper, logo etc.).
As you can see in the above image, I am trying to extract three entities from the invoice:
- amount (for each line in the table)
- tariff (for each line in the table)
Now I actually don't need the table, but what seems to be common across all my supplier invoices is that the
tariff for line imtes, is always within a table.
My idea of a pipeline is:
- Get all invoices (PDF files) and convert each page to a PNG image.
- Label all the individual pages, using a label annotation tool, to label
tariff- including the bbox information for each label.
- Train a model using the above dataset.
My questions are:
- Does this seem like a good approach to such a problem?
- Do I have to split up the model into two? With this I mean, does it make more sense to first identify the
tablewithin an image with one model, then serve the table image to another model, and extract the
amountwith the 2nd model?