0
$\begingroup$

I've got a multi-step pipeline that produces output which is then (sometimes) rated by users. Something like this:

  1. Run sentiment analysis on input
  2. Run intent analysis on input
  3. Choose gating weights of mixture of experts
  4. Produce raw output from mixture of experts
  5. Refine with sentiment

Each of these five steps contributes some amount to the success/failure of an output. With users only rating the final output, how do I figure out how much weight to assign each step for correction, like this:

Final Rating: 4 stars

  1. Run sentiment analysis on input - 20% impact on rating
  2. Run intent analysis on input - 30% impact on rating
  3. Choose gating weights of mixture of experts - 15% impact on rating
  4. Produce raw output from mixture of experts - 25% impact on rating
  5. Refine with sentiment - 10% impact on rating

Any papers/ideas digging into how one might go about doing this?

$\endgroup$
1

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.