Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm using SciKit-Learn's OneHotEncoder to encode some categorical features. However, the time complexity of the function seems to have an exponential dependence on the number of rows passed to it.
I would like to profile my code to figure out what's going on, particularly, I would like to use cumulative time to rank the profiler's output. I'm using Jupyter notebooks to write the code, how can this be achieved?
Jupyter has built in "cell magics", one of those cell magics is the prun function.
To use it, you put a call to prun at the top of the cell. For example:
%%prun
YOUR-CODE-BELOW
The %% at the beginning of the command tells Jupyter that this is a Cell Magic, if you don't include this, you'll get a syntax warning. To rank by cumulative time, you use the -s option and pass it "cumulative time":
%%prun -s "cumulative"
YOUR-CODE-BELOW
This will profile the code in your cell and show you the output in a pop-up box on the bottom of your screen.