One key component of MLOPS is continuous training. Which means the end to end training is put in a pipeline which can be triggered, versioned and metadata of the pipeline can be tracked. Thus enabling retraining of the model without lots of manual efforts. Which tool/package do you use for creating such a training pipeline? I am looking for simple tool with following criteria
- Using only python (i.e. docker is not mandatory like kubeflow)
- Dev have lot of flexibility and is not created for specific tasks like nlp, regression, classification etc.
- Little learning curve/new syntaxes but is simple to understand and use
- Platform or cloud agnostic
I came across the below and didn't find it very useful due to the mentioned reason
- Kubeflow - every component of the pipeline need to be in a separate docker
- Kedro - require learning lots of new syntax etc