Where to find statistically relevant documentation of common Python packages?

Question

I am trying to entice my lab to transition from Matlab and R to Python. The main objection at this point seems to be that Python's analytical libraries are not sufficiently well documented. Given how prolific Python is, I suspect that sufficiently detailed documentation exists and we simply cannot find it.

Example

Recently, I needed to upsample a signal (1D vector) and found a Python function called resample. In contrast to Matlab and R, pandas documentation doesn't tell me what kind of interpolation resample uses (linear, cubic splines, pchip?).

Is there any place where I could find such information about this and other libraries, preferably with equations or references to papers? I understand that I could analyze source code but this isn't most time efficient. If you know R, I am basically looking for a Python equivalent of CRAN (PDF warning).

Thanks!

Answer 1

Switching to Python from Matlab is going to be somewhat dependent on the field you are in (for example, if you are in image processing/ imaging, Matlab is pretty well solid and so swithcing will be more difficult) and the stubbornness of your coworkers/boss. While documentation for Matlab is easy to find, it's also all you get, whereas Python packages generally have decent documentation as well as countless posts on SE in which someone has likely already answered your question.

One of the greatest aspects of Python is that it is open source, but it can make finding the right tool slightly more time-consuming. Your example is about resampling. Pandas resample is designed for time-series (see pandas doc). All the arguments are described, but I can also look at the source code if I really need to understand anything more in depth. Or I search for some examples online: http://benalexkeen.com/resampling-time-series-data-with-pandas/

But it seems like you are actually interested in interpolation functions. If we search for "Python interpolation" we will discover a few methods. Looks like Scipy has an extensive package for interpolating: https://docs.scipy.org/doc/scipy/reference/interpolate.html

It takes some time to figure out how to search for what you need in Python, but once you get the hang of it, there are a lot of great examples online of all sorts of application. Plus, plotting in Python (and R) is way beyond Matlab (c'mon mathworks). In the end, you can do what I did which is flat out refuse to use Matlab unless it is actually necessary and hope they don't fire you!

Answer 2

I think the documentation, at least for your example, is not too obscure. It doesn't tell you what kind of interpolation it uses because it doesn't use any: resample is a deferred operation. In order for it to work you have to call it in conjunction with a function that performs the interpolation, e.g. series.resample(frequency).mean() or series.resample(frequency).interpolation('cubic') (as can be seen from the examples in the documentation). Sometimes, it can be helpful to also look into the release notes for additional information.

Scipy and Numpy are sometimes not very detailed, but they contain at least a decent number of references with more information. (There is also a section on Matlab versus Numpy here. If you are familiar with dplyr in R , then the pandas cheat sheet can help as a quick comparison between R and pandas for typical data wrangling operations.)

I'm not aware of a "second more detailed documentation" that encompasses the entire libraries. You probably have to fall back to books (which, of course, aren't complete or always up to date).