MLOps for beginner

Question

I am 1 year old in ML and have been using jupyter notebook to build static models all these days, do some analysis and present my results to the bosses as it was all POC.

Now, we would like to scale the solution to become automatic and be able to feed the real data stream automatically and allow model to learn automatically without me doing batch based update.

Since, all this is completely new to me and am not a software developer/engineer. Can you help me with the below queries

a) Is there any online courses/institutes/books for beginners like me?

b) Is there any python packages that can allow for online learning of models and update the results etc? Or what are the list of packages that I can refer for MlOps purpose?

c) I would like to learn via tutorial of IRIS dataset etc. Where they can walk us through how once model is built, it is taken to production, handling preprocessing of future data inputs etc.

Answer 1

a. For a beginner I would suggest the fullstackdeeplearning course, it's a modern overview of tools and best practices for ML in production. As you can see below, there are a lot of moving pieces.

b. What you are asking for can be done with Spark + Airflow. In particular Airflow (or similar tools such as Luigi) allows to create very customised data pipelines. The learning curve is a bit steep, but there are good resources available online.

c. The course above should answer your questions, as the data side is not really deep learning specific, but can apply also to data-science workflows.

Answer 2

You can do live learning but most models don't require it, because many businesses don't need to learn directly from new input.

Nevertheless, you can apply an automated task every time range (day, week,...) that learns from new input collected during that time range.

Depending on the business, old data could be necessary for the learning process if new data is too scarce.

Then, you should include quality checks (model accuracy, test results, performance, etc.) to ensure that your model will work well in production.

As production is a 0 risk environment, the first automated learning should be carefully supervised and manually validated, before becoming fully automated.