I'm working on a automated ingestion system which takes a PDF or doc file or a URL. It then parses the file and get me the required text in a json format but there are some error and there are few things that I want to capture differently or add.

As to give you an idea what I really want to do is: Let's say I've a URL of Step by Step Instructions on doing something. I want to scrape/parse this web-page and convert these instructions into smaller and meaningful instruction, so Amazon Alexa or any other voice assistant can speak and give the user instruction through voice.

The output I'm getting by scrapping this webpage is not in proper format. Till now I've been manually improving by adding the desired words and punctuation. But the number of these URL will increase over time so it is not possible to give so much time on improving the instructions by using a human to add the missing words or break a sentence into two meaningful instructions. So how can I automate this part using ML/DL so that this system can be scaled.

So can you guide me on how can I leverage Machine Learning to improve this ingestion system?

Thanks in advance. Let me know if you have any questions regarding this or If I've not made something clear which is required to answer it correctly!!!

  • $\begingroup$ I think that the scope of what you are asking is too big. You should focus your question on some specific data-science problem you have $\endgroup$
    – Mark.F
    Dec 27 '18 at 13:57
  • $\begingroup$ I understand. But the issue with this problem I'm facing is that there is not enough literature available that can guide me towards, which kind of solution will work for this problem. $\endgroup$
    – Khushhalm
    Dec 27 '18 at 22:44
  • $\begingroup$ 1. Get labelled data - you need examples of step by step urls and the chunks of page for a step labelled. 2. Use ML to identify likely steps on a page - could use raw html, the text only, a combination, LSTMs, etc. Basically want to classify a paragraph on a page as "step" or "not a step". 3. Order the steps appear are your steps. 4. Iterate. $\endgroup$
    – Ken Syme
    Dec 28 '18 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.