10
$\begingroup$

In the past, when trying different machine learning algorithms in order to solve a problem, I used to write drown the set of approaches on a notebook, keeping details such as features, feature preprocessing, normalization, algorithms, algorithm parameters... therefore, building a hand-written logbook.

However, currently I'm concerned about using a 'more professional' tool, so that I can keep more details and even share it with other team-members, who are also able to stamp their approaches.

It would be great an automated and collaborative tool that keep track of the work done, considering details like: features, algorithms, algorithms parameter, data pre-process, data, metrics... beyond a collaborative Google Drive Spreadsheet for instance.

How are you solving this? How are you keeping track of the work done? What's your logbook tool?

Thank you very much in advance.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
6
$\begingroup$

How are you solving this? How are you keeping track of the work done? What's your logbook tool?

This might not be the best approach. But, this is how my team does it. We believe that for pulling off an end-to-end data science experiment, proper conscience is very important. So, we use Slack for the same for our discussions and the meetings.

In addition to them, we have Rmd (R markdown) files for documenting the planning and the analysis parts.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I like the idea of using markdown, however, I guess it's not the best approach/solution. It's good for a starter point. I also use Slack, and I highly recommend it for teams coordination and meetings. $\endgroup$
    – Jorge
    Commented Apr 14, 2016 at 14:53
  • $\begingroup$ @Jorge Yeah, I agree that it's not the best approach :) However, I haven't found a good alternative worth spending on! $\endgroup$
    – Dawny33
    Commented Apr 14, 2016 at 14:58
3
$\begingroup$

Check this out, looks like exactly what you need

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is this a Google product? The UI looks very google-like $\endgroup$
    – Dawny33
    Commented Apr 11, 2016 at 11:42
  • 1
    $\begingroup$ I don't know. I am not associated with that site in any way. They have their owners and hoster information listed on the front page though. $\endgroup$
    – Diego
    Commented Apr 11, 2016 at 11:49
2
$\begingroup$

How are you solving this? How are you keeping track of the work done? What's your logbook tool?

For my bachelors thesis (write-math.com) I wrote my own little toolkit to go through different models / preprocessing steps very fast. Each experiment had one configuration file (see hwr-experiments repository). For example:

data-source: feature-files/baseline-3-points
training: '{{nntoolkit}} train --epochs 1000 --learning-rate 0.1 --momentum 0.1 --print-errors --hook=''!detl
    test {{testing}},err=testresult_%e.txt'' {{training}} {{validation}}
    {{testing}} < {{src_model}} > {{target_model}} 2>> {{target_model}}.log'
model:
    type: mlp
    topology: 24:500:369

The trained model is stored; it is pretty fast to get the evaluation results (e.g. accuracy, confuscation matrix).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.