1
$\begingroup$

I am a data analyst and use jupyter notebook for various data explorations. Since I often modify code a lot, one issue I constantly have is lost track of which code version a specific variable corresponds to.

Specifically, suppose I have two functions f1(), f2() and set values v1 = f1(), v2=f2(), later I update code for both f1, f2, and update v2=f2() but forget to update v1. Then after other bunch of edits, I have totally no idea which code versions v1, v2 correspond to. Although I could just "restart and rerun" the notebook, in a lot of cases, it is just too expensive to do so. So I wonder if it is possible to somehow keep track of what version of code a specific variable value is corresponding to? It would be even better if we can keep track of all historic values of v1, v2.

Not sure if this is doable, would like to hear all ideas and suggestions. Thanks in advance!

$\endgroup$

1 Answer 1

3
$\begingroup$

There are probably some jupyter notebook plugins like this that can do that but it seems very expensive to keep track of the history of every object update made in a big notebook (which seems to be your case).

A suggestion I would make is to try to keep a more organized approach to you notebook development. Notebooks are a great tool to develop code in the process of data analysis and exploration but if you are creating processing steps that should be applied on your data, data scrutures or models maybe it is easier to develop this code in a separate .py script and import the function inside your notebook. This way you are always using the same version of your function and changing it would change every time you apply it (of course the problem of having to run the whole notebook again could appear).

$\endgroup$
1
  • 1
    $\begingroup$ I agree with this, processes should not be in notebooks, they're mainly good for EDA and analysis. Scripts with your processes can be imported in the notebook and run from there to visualise results and play with the output you return. $\endgroup$ Commented Nov 8, 2023 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.