2
$\begingroup$

I am in the research phase of a long project and am willing to get some useful feedback from your side about the most appropriate project path to take.

Current situation:

  • A large team of so called production editors (PEs) is semi-automatically processing MS word files in order to do the copy-editing and layout.
  • This is currently semi-automatized with word macros where the PEs will apply different macros on different text paragraphs in order to set the appropriate style, position, font etc.
  • Still many actions are manual and depending on the human eye checks, resulting at the end many mistakes and an average quality

Plan

  • Start using machine learning, later maybe neuronal networks, statistics probability and artificial intelligence
  • The usage of machine learning (to start with) could be in the way that the system will learn the manual tasks done by the user (meaning the tool was not able to do those correctly in the automatized way) and then try to apply those in next similar situation
  • The term of similar situations is important here as this will imply natural language processing (NLP). The staff is dealing with word documents that could contain anything (within a similar layout)

The dilemma of the day is :) :

Where to start from in order to do some use case analyses? What I currently have:

  • More than 100k word files that are correctly done (except the user mistakes that may still lay around)
  • A list of rules that apply to the word documents describing the exact styles that needs to be used (non exhaustive)
  • Basic Machine Learning knowledge (starting with AI studying)
Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Starting with a small chunk of documents (~50) as a training set with a set of rules implements would be a good starting point. Train using a few algorithms and see if your training accuracy is coming out as acceptable or better than the human accuracy.

Then slowly increase your training batch size and see the impact on the results.

Once you believe that the training set is a partially good representation of the dataset, go ahead do a test run. Then keep growing from there.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you Bonson, but how to indentify those algorithms? Any preferred coding languages for this? Imagine I have a word file, what to do with it? Convert to html? Keep it as is and run VBA macros to extract something? $\endgroup$
    – Miloš
    May 14, 2018 at 11:16
  • $\begingroup$ I use python. However, you can use any language, Java, R, etc. Not sure if VBA can be used. As a starting point, maybe converting the word file to text files and using them as input might help. Also, if you are just implementing multiple algos using an api and seeing the results might help. $\endgroup$
    – Bonson
    May 15, 2018 at 0:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.