2
$\begingroup$

I am working on a real-time recommender system predicting a product to a user using deep learning techniques (like wide & deep learning, deep & cross-network etc). Product catalogue can be huge (1000s to 1 million) and for a given user, the model needs to be evaluated against each product in real-time. As scalability is an important concern, is there any way to reduce the serving time complexity by tuning model architecture?

$\endgroup$
0
$\begingroup$

You write ... the model needs to be evaluated against each product in real-time., which gets me thinking that you use a binary classification (sigmoid in the final layer) architecture with negative sampling for the user/item interactions when training your model.

Have you considered using multi-class classification instead? Thus, for the user only predict once for the entire product catalogue, and selecting the top-k candidates from the softmax-layer. This way, you only need to feed-forward once through your neural net during inference.

$\endgroup$
3
  • 1
    $\begingroup$ the network predicts click-through rate (CTR) for each product and we choose the top CTR products to show to the user. Product_id is an input to the network which makes multiple evaluations against multiple product_ids during inference. I am not sure whether softmax can handle such a scenario. $\endgroup$
    – Aljo Jose
    Jan 12 at 13:25
  • $\begingroup$ If "the network predicts click-through rate (CTR) for each product...", then you only need to feed-forward once during inference right? Is this too slow for real-time you mean? $\endgroup$
    – Marcus
    Jan 12 at 16:41
  • $\begingroup$ it is a feed-forward, but for a given request data, CTR against each item needs to be computed. if there are 10K products, I would need to prepare a matrix of 10K rows with same request data changing product ids and response needs to be in less than 100 milliseconds. $\endgroup$
    – Aljo Jose
    Jan 12 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.