Deciding tools to use for ML

Question

I want to setup a decent framework for building and deploying ML code on our servers. The server side code is all java. I have inherited a system that uses weka as it is based on java and makes life easy for the server side folks. But data scientists who come in, use R/python to do their modelling and then write some glue code to make the models run in java environment on weka. This library is getting outdated(works alright though), but mainly writing this glue code is getting cumbersome/repetitive and can be done away with. I'm looking for an optimized pipeline that helps me get models deployed quickly.

How is this done in the industry? What does your pipeline look like?

I would like to be able to quickly model using one language that data science team uses(I'm tending to python) and be able to deploy/call these models for prediction on the server side without having to redo a bunch of stuff to get it working there.

Answer 1

Well, I think your question does not have a "correct" answer since you are looking for a piece of advice on ML model deployment.

As you mentioned, you have a complete back-end in java, and you should not drop it. Instead, (1) you can take advantage of a micro-services oriented approach. With micro-services, you can communicate different pieces of code using HTTP. You may have some of your back-end controllers in java, and some kind of ML-managers to wrap prediction routines with web routes.

I think it's a good idea for you to learn python, and also python web frameworks (flask is a very simple one, you can google "ML deployment flask" or something like that, and there is plenty of tutorials). Also, (2) you can convince your data scientists to write their micro-services for you because it is simple to do. Then, you can get their code and adapt to better look like a web-service.

Another way to take advantage of your java back-end is to (3) try to translate the python/R ML code to Spark MLlib, which has many powerful methods for you to write your java code. However, you will eventually have to code some operations that your data scientists made in python/R, which is not available in MLlib.

It depends, but I hope that it helps!