5
$\begingroup$

I am using tensorflow-serving to write a server to consume models in production. I have a question about consuming the service by clients: does tensorflow-serving support a REST API? Is there is anyway to modify it?

I have checked several github projects: here, here, and here.

$\endgroup$
  • $\begingroup$ Isn't the first link what you want? $\endgroup$ – Icyblade Mar 10 '17 at 11:22
  • $\begingroup$ Yes it is very similar, I am asking if the same is provided by google, I like to use the tensor - server . It is ver efficient. $\endgroup$ – Bashar Haddad Mar 10 '17 at 14:17
  • $\begingroup$ This is maybe what you are searching for: becominghuman.ai/… $\endgroup$ – Christian Frei Apr 12 '18 at 8:09
0
$\begingroup$

I have an answer now for my question. I will share briefly the main steps / technologies I used to deploy the model in production.

I am using Python programming language. After training and generating valid models I wrote a restful api using Python programming language and flask.

Using flask you can write a restful api. Three important points:

1- It is very important to give attention to where you will define the model architecture/initialize the parameters/ define the session. Avoid doing this each time you call the restful api. this will be very expensive.

2- Flask provide a very good mechanism to run servers in production environment. Read about flask + wcgi Avoid runing the server code (the resful api) directly, in this case you will not have direct and full control.

3- Watch the memory and the cpu usage, make sure to limit the maximum number of instances that can run in parallel. Remember these models can take a lot from the memory.

unfortunately, I can not share codes with public. But hopefully my answer can give an idea about how to do it in production.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.