0
$\begingroup$

I've couple of recommendation algorithms that I want to A/B test. Algorithm A has 90% user coverage and algorithm B has 95% user coverage. That means if the algorithms are asked to provide recommendations for 1000 users, algorithm A can give it for 900 of the users and algorithm B can give it for 950 other users. Say for example out of these 1000 users 87% has recommendations from both algorithm, 3% has recommendations from only algorithm A and 8% has recommendations from only algorithm B. What would be the ideal approach to bucket the users

  1. Bucket only those 87% users as that can generate even bucketing, but the higher coverage of algorithm B won't reflect in A/B test.
  2. Bucket all 95% users, but that could cause uneven bucketing.
$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.