26
$\begingroup$

I originally came from R, but Python seems to be the more common language these days. Ideally, I would do all my coding in Python as the syntax is easier and I've had more real life experience using it - and switching back and forth is a pain.

Out side of ML type stuff, all of the statistical analysis I've done have been in R - like regressions, time series, ANOVA, logistic regression etc. I have never really done that type of stuff in Python. However, I am trying to create a bunch of code templates for myself, and before I start, I would like to know if Python is deep enough to completely replace R as my language of choice. I eventually do plan on moving more towards ML, and I know Python can do that, and eventually I would imagine I have to go to a more base language like C++.

Anyone know what are the limitations of Python when it comes to statistical analysis or has as link to the pros and cons of using R vs. Python as the main language for statistical analysis?

Improve this question
$\endgroup$
3
  • $\begingroup$ There are plenty of people doing this in Python, so you've already proven it's viable. Related: R vs P for longitudonal data, for data science, for machine learning and performance differences. $\endgroup$
    – Mast
    Commented Jun 29, 2020 at 12:48
  • $\begingroup$ I personally use R but Python can do just as much. IMO the main point of Python over R is that Python allows you to be more versatile if you plan to do something else than stats or data management. $\endgroup$
    – Gainz
    Commented Jun 29, 2020 at 16:16
  • 2
    $\begingroup$ Having moved from R to Python, one thing I miss a lot is R-Studio. I use PyCharm which IMO is not as (noob)user friendly. I know R Studio is working to integrate Python but a few months ago it was still not on par with the R implementation. $\endgroup$
    – wotter
    Commented Jul 1, 2020 at 14:16

7 Answers 7

Reset to default
34
$\begingroup$

Python is more "general purpose" while R has a clear(er) focus on statistics. However, most (if not all) things you can do in R can be done in Python as well. The difference is that you need to use additional packages in Python for some things you can do in base R.

Some examples:

  • Data frames are base R while you need to use Pandas in Python.
  • Linear models (lm) are base R while you need to use statsmodels or scikit in Python. There are important conceptional differences to be considered.
  • For some rather basic mathematical operations you would need to use numpy.

Overall this leads to some additional effort (and knowledge) needed to work fluently in Python. I personally often feel more comfortable working with base R since I feel like being "closer to the data" in (base) R.

However, in other cases, e.g. when I use boosting or neural nets, Python seems to have an advantage over R. Many algorithms are developed in C++ (e.g. Keras, LightGBM) and adapted to Python and (often later to) R. At least when you work with Windows, this often works better with Python. You can use things like Tensorflow/Keras, LightGBM, Catboost in R, but it sometimes can be daunting to get the additional package running in R (especially with GPU support).

Many packages/methods are available for R and Python, such as GLMnet (for R / for Python). You can also see based on the Labs of "Introduction to Statistical Learning" - which are available for R and for Python as well - that there is not so much of a difference between the two languages in terms of what you can do. The difference is more like how things are done.

Finally, since Python is more "general purpose" than R (at least in my view), there are interesting and funny things you can do with Python (beyond statistics) which you cannot do with R (at least it is harder).

Improve this answer
$\endgroup$
4
  • $\begingroup$ Sounds great. I will use Python then as I don't mind installing more packages. Plus it makes it easier to interact with other platforms, which might have Python based APIs and what not. $\endgroup$
    – confused
    Commented Jun 30, 2020 at 3:47
  • $\begingroup$ @confused see this Github for some basic Python ML github.com/Bixi81/Python-ml $\endgroup$
    – Peter
    Commented Jun 30, 2020 at 20:03
  • $\begingroup$ Relatedly, base R has usually one way to do things, whereas Python can have several methods. For instance, computing a mean can be done with statistics.mean(), numpy.mean(x), scipy.stats.mean(x), to mention only the obvious ones. I find this a bit of a problem for stats students who are not into coding. $\endgroup$
    – PatrickT
    Commented Jul 2, 2020 at 5:19
  • $\begingroup$ One more point: The built-in features in R are really basic. As soon as you move beyond basic statistic, you'll also need to install additional packages (MASS, caret, ROCR...). And that's where the trouble begins: There is a whole jungle of packages in R, many doing similar, but not exactly the same thing, and varying widely in their quality. Finding a good package can be really frustrating. In Python, there is only a handful of packages you'll need and they are much better curated. $\endgroup$
    – Igor F.
    Commented Jul 2, 2020 at 6:59
13
$\begingroup$

Python being more widely used is an important consideration. This will especially become important when applying for a job. Also Python has as many if not more key statistical and ML/AI tools as R, and a larger open-source base to utilize. Python is designed for programmers, R is designed for statisticians. Originally I was a R programmer, but most of my colleagues were using Python so I eventually switched over.

Here are some of the basic differences:

Python:

  1. programmer friendly
  2. debugging easier
  3. More open-source support (stack web sites, etc)

R:

  1. Easier and simpler to write scripts
  2. Works better with other languages
  3. More built in functionality

Good reference to check out: datacamp.com/community/tutorials/r-or-python-for-data-analysis

Also should mention that i have used R code within Python, using Rpy2. If you are using a notebook, just use %%R, after installing the necessary R libraries

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ There is also reticulate for using Python in R / Rmarkdown notebooks. $\endgroup$
    – AlexR
    Commented Jun 29, 2020 at 18:08
  • $\begingroup$ Good to learn about this. I have built several data flows between R, Python, Spark, SQL, etc in my previous job, but only using R within Python, not the other way around. Using Python within R can open up several more possibilities. $\endgroup$
    – Donald S
    Commented Jun 30, 2020 at 3:06
  • 1
    $\begingroup$ Interesting that you can use R within Python and Python within R. That makes life a lot easier. $\endgroup$
    – confused
    Commented Jun 30, 2020 at 3:48
5
$\begingroup$

I'd like to add two points to the existing answers:

  • There is excellent interaction between R and python, with various possibilities for either direction.

    To me, it's not that much of a decision python vs. R. The decision is to choose the main language appropriately for the project at hand, and then do parts in the other language if that is better for some reason.

  • I find the facilities to generate reports much more convenient in R.
    Since lots of my work consists in producing reports about statistical analyses, I mainly use R.

    To the point that were I to encounter a data analysis + report today that I think is better done in python, I'd set up the report as "R"markdown and do the python in python chunks.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I'd like to add that in my experience integrating python into R is much easier than the other way around. Rstudio and notebooks can integrate python chunks directly, reticulate allows you to use almost rat python in your R code, etc. Calling R from python is much more involved and not as straigth-forward. $\endgroup$
    – Fnguyen
    Commented Jul 1, 2020 at 10:35
5
$\begingroup$

One thing that can be a gotcha coming from R to Python is that the Python stats ecosystem tends to be more machine learning-ey oriented rather than inferential stats-ey oriented.

This can create some hiccups, because some of the defaults in R that are the defaults because people who do inferential stats like in the social sciences always use them, are not the defaults in the main Python libraries.

For example, Statsmodels, one of the standard libraries for inferential stats, doesn't include the intercept by default when you do linear regression, UNLESS you use the R-style formulas with Patsy, in which case it is included.

Another example: Scikit-learn in Python uses the divide-by-n ("population") formula for standard deviation, while R uses the divide-by-n-1 ("sample") formula.

Those sort of things tend to be really confusing for people new to the ecosystem, and create totally unnecessary cognitive burden. So that's a tradeoff.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Another example: skLearn/scipy.linalg.lstsq and R's lm handle rank-deficient model matrices in very different ways ... $\endgroup$
    – Ben Bolker
    Commented Jul 1, 2020 at 22:37
3
$\begingroup$

I eventually do plan on moving more towards ML

One aspect that I would like to add based on what I observed.

Things are moving with more focus towards Deep Learning e.g. Neural Networks and in this space, most of the dominating Libraries supports Python as first choice.

Companies manage a separate Python version to open-source, just to maintain the user base even though they themselves use either a C++ compiled version or something different. It's because of the two-way-additive process i.e. since Python has gained fame, companies are creating an open-sourced framework/library in Python and easily available Frameworks/Libraries are attracting more users.

Stackoverflow 2019 Survey

Most Popular Technologies - Python - 41.7% $\hspace{1cm}$ R - 5.8%
Other Frameworks, Libraries, and Tools - Pandas and Tensorflow are in top 5-6
Most Wanted Languages - Python is at the top with 25.7%
Most Wanted Framework - Tensorflow at 2nd after NodeJs

Same logic goes with Books/Blogs and Tutorials.
I will agree that concepts don't change with a programming language but the examples/code provided in the books/blogs definitely accelerate the learning.
Almost everyone in the Industry will recommend this book to a beginner and I also found it the best.
Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems 2nd Edition by Aurélien Géron

Improve this answer
$\endgroup$
1
$\begingroup$

For the love of the flying spaghetti monster, use anaconda to install the needed packages for data science. I have seen both Python and R being used in the data science setting and both needed additional packages to execute any data science capabilities. Conda made it way easier to install them.

From my point of view, Python has a better support for all kind of packages. There are simply more ports to Python than to R, but this may change in the future.

https://docs.conda.io/projects/conda/en/latest/user-guide/install/
conda install scikit-learn
Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm not a fan of Anaconda. Eventually you get to the point where there are issues install packages through Anaconda. Just install Python 3.6, isntall jupyter notebook separately, and R studio separately. Same functionality, a lot less bulk. $\endgroup$
    – confused
    Commented Jul 1, 2020 at 8:56
0
$\begingroup$

As others have pointed out, python is more general, more programmers oriented, with more libraries and better hardware support. I'm not an R user, but python seems faster (c based) and more suitable on processing large files, or extracting big data from sql, most times in my experience is a previous step before apply statistics or AI to data.

Of course if you try processing using Dataframes and all data artifacts R like, with pandas or other math libraries, you end with a bad performance as in R. But with python you also have the option to process raw data files, line to line and byte to byte, and optimize processing time on big data sets, use multiprocessing for full machine use, etc.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.