I need some help, i am working on a problem where i have the OCR of an image of an invoice and i want to extract certain data from it like invoice number, amount, date etc which is all present within the OCR. I tried with the classification model where i was individually passing each sentence from the OCR to the model and to predict it the invoice number or date or anything else, but this approach takes a lot of time and i don't think this is the right approach.

So, i was thinking whether there is an algorithm where i can have an input string and have outputs mapped from that string like, invoice number, date and amount are present within the string.

E.g: Inp string: the invoice #1234 is due on 12 oct 2018 with amount of 287 Output: Invoice Number: #1234, Date: 12 oct 2018, Amount 287

So, my question is, is there an algorithm which i can train on several invoices and then make predictions?


1 Answer 1


Keras functional API's are a way your can solve you problem. Using keras functional API, we can build models that resembles more like graphs such as this:

enter image description here

In order to build a model like this, you can use keras as follows:

from keras.models import Model
from keras import layers
from keras import Input

input_layer = Input(shape=(100,), dtype='float32', name="Input")
split_layer = layers.Dense(32, activation='relu', name='split_layer')(input_layer)
first_layer = layers.Dense(32, activation='relu', name='first_layer')(split_layer)
second_layer = layers.Dense(32, activation='relu', name='second_layer')(split_layer)
model = Model(input_layer,[first_layer, second_layer])

In order to compile this model, we can define different loss functions for different layers

            loss={'first_layer':'mse', 'second_layer':'binary_crossentropy'},

Once you are done with building the network, you could simply fit you data as follows:

        {'first_layer': first_layer_Y,
         'second_layer': second_layer_targets},
  • $\begingroup$ Thank you for your answer but, how can input a string directly onto the keras, and how to prepare my dataset like the one i mentioned above? I'll have to preprocess my input string through some vectorizer and then encode the target values. How to do all that so my output is mapped from within the input string itself $\endgroup$ Commented Oct 10, 2018 at 6:42
  • $\begingroup$ If the data is in images, I cannot suggest you anything other than OCR to extract the text. Once the text is extracted, you can solve your problem by either Building attention models or Using IOB tagging etc. If this is not helpful, kindly rephrase your question here in comment section properly. $\endgroup$
    – thanatoz
    Commented Oct 10, 2018 at 6:55
  • $\begingroup$ I have done the OCR of the image and the text of the image. Now i want to extract certain data from that OCR'ed text like invoice number, amount and date. So, i want to make a model where as input i have the OCR of the image and for output i want to map the invoice number, amount and date from the OCR input itself. $\endgroup$ Commented Oct 10, 2018 at 6:59
  • $\begingroup$ Try the attention model and try to converge the model to classify the required data ( invoice number, amount and date ). $\endgroup$
    – thanatoz
    Commented Oct 10, 2018 at 8:15
  • 1
    $\begingroup$ OKay, I looked into it and found encoder decoder sequence to sequence with attention model and I think it might be a possible solution. Thanks a lot. $\endgroup$ Commented Oct 10, 2018 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.