2
$\begingroup$

We are having this requirement of extracting information from a credit history document. Usually it is a PDF and a computer generated document. Because these PDFs are generated by different sources, the layout of the document will be different for each source. The columnn header labels will also be different. Presently, there are 4 sources which are generating this document, but going forward, it will be from many sources. From each of these documents, we will need to extract information such as lender name, lending amount, outstanding balance etc;

I need to know what are the steps and practical approach involved in extracting the data I want such as lender name, amount, balance etc;

Do we have an established Machine Learning / Deep Learning approach that can be implemented here? Just getting to know the basics of ML/DL, therefore need a direction please

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

The task is doable, but time consuming and not easy. This is how I would plan the work:

  1. Write a PDF scraper that navigates the document and converts all the informations it contains into some standardized textual format.

  2. Label the words/elements of this text that correspond to the informations that you are looking for ("lender name", "lending amount", "outstanding balance", etc.).

  3. Run some NLP model, such as an RNN classifier or CRF, to extract information from text.

Steps 1 and 2 are very time consuming (2 more than 1), but it's certainly doable. It will be a lot of work, especially labeling observations to create the Training set, but a nice thing to put on your CV.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you. Will give it a go $\endgroup$
    – Rajesh Sharma
    Commented Jan 9, 2020 at 2:12
  • $\begingroup$ a little elaboration will help. The plan helped me get a high level understanding. But certainly not enough for a hands-on trial. Step#1, I would use textract. Step#2, I believe is the training dataset. But not sure how the dataset looks like $\endgroup$
    – Rajesh Sharma
    Commented Jan 22, 2020 at 11:07
  • $\begingroup$ Your dataset could be organized as for a classical RNN classifier task. Treat each document/observation as a text, a sequence of tokens, and train a classifier to classify each of these tokens as ('lender name', 'lending amount', 'outstanding balance', nothing, etc.). $\endgroup$
    – Leevo
    Commented Jan 22, 2020 at 13:39
  • $\begingroup$ is it possible for you to point me to a sample dataset that will be similar to the dataset that we will need to prepare for our problem? Thanks a lot for the advice so far. $\endgroup$
    – Rajesh Sharma
    Commented Feb 5, 2020 at 14:44
  • $\begingroup$ Unfortunately I never worked with similar data, and I don't know of existing public datasets on this topic. Aanalogous tasks are about extracting relevant information from applicants CVs. I know the data are not the same, but the nature of the task and the models' architectures are extremely compatible. Are you still interested? $\endgroup$
    – Leevo
    Commented Feb 6, 2020 at 8:54
0
$\begingroup$

Since I understand you only need textual info, I would use a text printer to print the PDF doc to a text file.

After that I would extract the body text and discard the headers and footers. Maybe a simple code could do it based on the number of empty lines and the fact those text blocks repeat page after page. I would deal if the fact some documents could have a multicolumn layout.

After having the main textual content I would go for a DL model to extract the content I need, assuming only a DL model is able to do it.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.