2
$\begingroup$

I have an application that receives ~10k of requests per day; each request has multiple parameters and goes through a pipeline with multiple steps and they finish in about 1 hour. Let's say the requests are like bake_cake [flavor=chocolate] [topping1=strawberry] [topping2=cream]

Given a history of requests from previous days, is there a ML framework that can help me predict the "100 most like requests to arrive today" so I can cache their results? Or some other similar strategy that can help me delivering part of those requests faster?

Ideally it would be based on how much I gain by delivering faster, how much I lose by processing a request that might not come, etc.; but for now even simpler algorithms could be of much help since currently there's no optimization.

$\endgroup$
1
  • 1
    $\begingroup$ I agree that you should start with simpler approaches. Why not simply start with calculating the X% most common requests from the previous days and cache their results? Then measure any improvements and go forward. $\endgroup$
    – Stergios
    Sep 5 '19 at 8:21
1
$\begingroup$

You need to validate your assumptions based on data driven analysis

  • Types of requests received on a day
  • Types of requests received on a holiday
  • Pattens for different times (Morning / Evening / Night)
  • Seasonality / Monthly / Weather based patterns

Post Analysis

  • Do you see correlation from Data & patterns
  • Do you see correlation from Data & Location
  • Do you see correlation from Data & Age / Gender / Locality

ML Models

  • Do you see correlation from Data & patterns
  • With the variables we can identity and model a forecast algo

ML Model is not the start. It is the outcome of detailed data and domain driven analysis

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.