Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am using tensorflow-serving to write a server to consume models in production. I have a question about consuming the service by clients: does tensorflow-serving support a REST API? Is there is anyway to modify it?
I have checked several github projects: here, here, and here.
I have an answer now for my question. I will share briefly the main steps / technologies I used to deploy the model in production.
I am using Python programming language. After training and generating valid models I wrote a restful api using Python programming language and flask.
Using flask you can write a restful api. Three important points:
1- It is very important to give attention to where you will define the model architecture/initialize the parameters/ define the session. Avoid doing this each time you call the restful api. this will be very expensive.
2- Flask provide a very good mechanism to run servers in production environment. Read about flask + wcgi Avoid runing the server code (the resful api) directly, in this case you will not have direct and full control.
3- Watch the memory and the cpu usage, make sure to limit the maximum number of instances that can run in parallel. Remember these models can take a lot from the memory.
unfortunately, I can not share codes with public. But hopefully my answer can give an idea about how to do it in production.
