1
$\begingroup$

We are having this requirement of extracting information from a credit history document. Usually it is a PDF and a computer generated document. Because these PDFs are generated by different sources, the layout of the document will be different for each source. The columnn header labels will also be different. Presently, there are 4 sources which are generating this document, but going forward, it will be from many sources. From each of these documents, we will need to extract information such as lender name, lending amount, outstanding balance etc;

I need to know what are the steps and practical approach involved in extracting the data I want such as lender name, amount, balance etc;

Do we have an established Machine Learning / Deep Learning approach that can be implemented here? Just getting to know the basics of ML/DL, therefore need a direction please

$\endgroup$
0
$\begingroup$

The task is doable, but time consuming and not easy. This is how I would plan the work:

  1. Write a PDF scraper that navigates the document and converts all the informations it contains into some standardized textual format.

  2. Label the words/elements of this text that correspond to the informations that you are looking for ("lender name", "lending amount", "outstanding balance", etc.).

  3. Run some NLP model, such as an RNN classifier or CRF, to extract information from text.

Steps 1 and 2 are very time consuming (2 more than 1), but it's certainly doable. It will be a lot of work, especially labeling observations to create the Training set, but a nice thing to put on your CV.

$\endgroup$
5
  • $\begingroup$ Thank you. Will give it a go $\endgroup$ – Rajesh Sharma Jan 9 '20 at 2:12
  • $\begingroup$ a little elaboration will help. The plan helped me get a high level understanding. But certainly not enough for a hands-on trial. Step#1, I would use textract. Step#2, I believe is the training dataset. But not sure how the dataset looks like $\endgroup$ – Rajesh Sharma Jan 22 '20 at 11:07
  • $\begingroup$ Your dataset could be organized as for a classical RNN classifier task. Treat each document/observation as a text, a sequence of tokens, and train a classifier to classify each of these tokens as ('lender name', 'lending amount', 'outstanding balance', nothing, etc.). $\endgroup$ – Leevo Jan 22 '20 at 13:39
  • $\begingroup$ is it possible for you to point me to a sample dataset that will be similar to the dataset that we will need to prepare for our problem? Thanks a lot for the advice so far. $\endgroup$ – Rajesh Sharma Feb 5 '20 at 14:44
  • $\begingroup$ Unfortunately I never worked with similar data, and I don't know of existing public datasets on this topic. Aanalogous tasks are about extracting relevant information from applicants CVs. I know the data are not the same, but the nature of the task and the models' architectures are extremely compatible. Are you still interested? $\endgroup$ – Leevo Feb 6 '20 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.