0
$\begingroup$

I have made a model to predict some values. In real time, it works perfectly fine and yields prediction within a fraction of a second. The algorithm I am using is PassiveAggresiveClassifier to predict on multiple classes.

However, when I load tested it, it lagged in performance. I tried on passing 500 requests at a time and it essentially bogged down.

Is there a way to enhance this aspect of a model program-wise? Can python solve this? Is there another way?

Thank you.

Improve this question
$\endgroup$
2
  • $\begingroup$ Do you mean 500 predictions requests using a pre-trained model, or re-training 500 times? That would make a massive difference since the training is by far the most complex part of the process. Also what do you mean by "in real time", i.e. what is the difference between your first test and your load test? $\endgroup$
    – Erwan
    Commented Apr 27, 2020 at 12:01
  • $\begingroup$ Yes, 500 requests per second on a pre trained model. Load testing is when you test your model's scalability on the server. The first testing is when you test the time of your prediction on one request locally which was 0.04 secs. $\endgroup$
    – Vaibhav Saxena
    Commented Apr 28, 2020 at 9:30

1 Answer 1

Reset to default
0
$\begingroup$

Assuming the server is as at least as powerful as the first local machine, you should obtain the same time or shorter duration by request. If not, there's a problem worth investigating.

Then to improve further it could be useful to design the server side architecture so that:

  1. the model is pre-loaded in memory, not read from file every time (huge bottleneck).
  2. there are multiple instances of the program running in parallel so as to distribute the load.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.