0
$\begingroup$

What are the main alternatives for prompt ops nowadays? By prompt ops, I mean a comprehensive solution for tracking prompt engineering experiments and also registering prompts in different stages, similar to how I would with an ML model in a model registry.

I went through MLFLow and it seems to have somehow what I look for, instead Sagemaker has tracking prompt experiments features, but it seems to allow prompt registration just using a "fake Estimator" object, that's not a very clean solution. Please correct me if there is another option for SageMaker I did not go through.

$\endgroup$
1
  • 1
    $\begingroup$ Not sure if it's exactly what you are looking for, but have a look at prompttools or agenta. You can find more LLM Ops resources in this github repo. $\endgroup$
    – noe
    Commented Mar 26 at 13:51

1 Answer 1

2
$\begingroup$

We use Langfuse to manage 100% of our prompts + us them for observability, evals, too. Excellent product & the team is very responsive to requests and it's open source if you care about that: https://github.com/langfuse/langfuse/

$\endgroup$
1
  • $\begingroup$ Thank you Jordie! It seems precisely what I was looking for! $\endgroup$
    – Raffaele
    Commented Apr 12 at 7:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.