0
$\begingroup$

What do you think of these steps? And where can I find help with this project?

I am in a business class and was assigned a data science problem. I was advised to seek out a coder at my school who can help, but that is not turning out well. Are there tutorials I can follow? Should I ask questions here step by step? Is there a freelance website where I can pay someone around $200 to do a very basic model? Is this even realistic for me to accomplish in 6 weeks?

This technical services company wants to automate their report writing. They want an algorithm that generates a rough draft report when you give it a set of documents.

From my research, I have found the following steps:

  1. Preprocess the documents that will be used to generate the report (remove headers/footers etc, tokenize text, handle data tables)
  2. Extract whole paragraphs of text and copy them into the appropriate section of an empty "Information Repository" document. Some examples of sections are "Site Description", "Previous Investigations", "Groundwater Sampling", etc. To do this, you would create a mapping between the sections in the "Information Repository" and keywords that indicate each section's content. Train a model (a text classifier like GPT or BERT) to classify paragraphs from the reports into relevant sections of the "Information Repository". Use supervised learning and provide labeled examples (each example is a paragraph along with its corresponding section label). With relevant sections now identified, extract the corresponding text from the documents and copy it into the appropriate sections of the "Information Repository". Evaluate and improve the model.
  3. Then part 3 I probably won't complete this semester, but it involves removing duplicate info from the Information Repository, and then extracting the relevant parts back into a new clean document which will be the rough draft (based on what type of report is being written, it will dictate what parts of the Information Repository is needed).
$\endgroup$
1
  • $\begingroup$ You can take a look at the NLP tutorial on Hugginface which has a pretty good ecosystem. $\endgroup$
    – Lelouch
    Commented Mar 24 at 17:09

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.