Machine learning model bundled with a library vs. an API

Question

I am thinking to "deploy" a machine learning model (in pickle it is sized 3 megabytes) and after discussing with my developer colleagues, they said it would be better if the model is packed as a python library instead of a microservice (like a rest API).

I wanted to ask what's your view on this: Pickled model packed in a library specifically meant for it vs. a rest API, pros and cons?

I was thinking that having it as a library could possibly be easier to use and wouldn't require to have to worry about deployment, web address etc.

Answer 1

The advantages of deploying the model as a package is the code will be part of the monolith application and require no remote calls thus:

• No additional operational demands
• No external dependencies
• Faster
• Fewer security issues
• Same uptime as the rest of the application

In general, most features stay in a single monolith application until there is are specific reasons to create separate services.

Answer 2

Complexity, just look at this simple deplyoment from fast ai. you get a web app in a couple of simple steps. Where as for restapi you need to know your way around a lot more libraries. For example the cool fastapi (flask+more capabilities). Example app

Online Learning there is no possibility to update (not trivial atleast) your model in the pickeled file. Because you restapi code is modularised this can be modified.

Overhead restAPi is designed to handle larger number of requests, and scale more easily.

Modularity/Control You define exactly how you interect with the client, more control but more complexity when using restAPI