I have made a model to predict some values. In real time, it works perfectly fine and yields prediction within a fraction of a second. The algorithm I am using is PassiveAggresiveClassifier to predict on multiple classes.

However, when I load tested it, it lagged in performance. I tried on passing 500 requests at a time and it essentially bogged down.

Is there a way to enhance this aspect of a model program-wise? Can python solve this? Is there another way?

Thank you.

  • $\begingroup$ Do you mean 500 predictions requests using a pre-trained model, or re-training 500 times? That would make a massive difference since the training is by far the most complex part of the process. Also what do you mean by "in real time", i.e. what is the difference between your first test and your load test? $\endgroup$
    – Erwan
    Apr 27 '20 at 12:01
  • $\begingroup$ Yes, 500 requests per second on a pre trained model. Load testing is when you test your model's scalability on the server. The first testing is when you test the time of your prediction on one request locally which was 0.04 secs. $\endgroup$ Apr 28 '20 at 9:30

Assuming the server is as at least as powerful as the first local machine, you should obtain the same time or shorter duration by request. If not, there's a problem worth investigating.

Then to improve further it could be useful to design the server side architecture so that:

  1. the model is pre-loaded in memory, not read from file every time (huge bottleneck).
  2. there are multiple instances of the program running in parallel so as to distribute the load.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.