I'm looking for a tool to track the results of several experiments/iterations in machine learning.

Training a model can take days/weeks, so it's essential to track its performance and be able to easily replicate results and roll back to previous versions. I'm looking for a tool that makes the process easy and streamlined like git did for version control.

Useful features would be:

  • open source with a license that allows commercial application
  • self-hosted [all the information should reside on computers we own]
  • python-friendly (ideally, tensorflow-friendly as well)
  • can store both hyperparameter values as well as references to specific datasets (the latter would be useful to track the effect of hard negative mining, etc.)
  • cluster-friendly
  • include basic plotting (it's useful to visualize learning curves)
  • automatic testing on multiple datasets
  • user tracking [like "blame" in git]

One such tool is Polyaxon. I'm using it extensively and it's really helping the experimentation pipeline.

It has features like experiment queuing, hyperparameter tuning, clustering, could be self-hosted or on the cloud, plots the metrics, it is independent of the framework and so on ...

I definitely recommend it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.