I'm looking for a tool to track the results of several experiments/iterations in machine learning.
Training a model can take days/weeks, so it's essential to track its performance and be able to easily replicate results and roll back to previous versions. I'm looking for a tool that makes the process easy and streamlined like git did for version control.
Useful features would be:
- open source with a license that allows commercial application
- self-hosted [all the information should reside on computers we own]
- python-friendly (ideally, tensorflow-friendly as well)
- can store both hyperparameter values as well as references to specific datasets (the latter would be useful to track the effect of hard negative mining, etc.)
- include basic plotting (it's useful to visualize learning curves)
- automatic testing on multiple datasets
- user tracking [like "blame" in git]