Information Extraction from image / text - approach?

Question

I need assistance with a ML project I am currently trying to create.

I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract 3 key elements from the invoices. These 3 elements are all located in a table/line items for all the invoices.

The 3 elements are:

• 1: Tariff number (digit)
• 2: Quantity (always a digit)
• 3: Total line amount (monetary value)

Please refer to below screenshot, where I have marked these field on a sample invoice.

I started this project with a template approach, based on regular expressions. This, however, was not scaleable at all and I ended up with tons of different rules.

I am hoping that machine learning can help me here - or maybe a hybrid solution?

The common denominator

In all of my invoices, despite of the different layouts, each line item will always consist of one tariff number. This tariff number is always 8 digits, and is always formatted in one the ways like below:

• xxxxxxxx
• xxxx.xxxx
• xx.xx.xx.xx

(Where "x" is a digit from 0 - 9).

Further, as you can see on the invoice there is both a Unit Price and a Total Amount per line. The amount I will need is always the highest for each line.

The output

For each invoice like the one above, I need the output for each line. This could for example be something like this:

{
    "line":"0",
    "tariff":"85444290",
    "quantity":"3",
    "amount":"258.93"
},
{
    "line":"1",
    "tariff":"85444290",
    "quantity":"4",
    "amount":"548.32"
},
{
    "line":"2",
    "tariff":"76109090",
    "quantity":"5",
    "amount":"412.30"
}

Where to go from here?

I am not sure of what I am looking to do falls under machine learning and if so, under which category. Is it computer vision? NLP? Named Entity Recognition?

My initial thought was to:

1. Convert the invoice to text. (The invoices are all in textable PDFs, so I can use something like pdftotext to get the exact textual values)
2. Create custom named entities for quantity, tariff and amount
3. Export the found entities.

However, I feel like I might be missing something.

Can anyone assist me in the right direction?