8
$\begingroup$

I'm currently in the very early stages of preparing a new research-project (still at the funding-application stage), and expect that data-analysis and especially visualisation tools will play a role in this project.

In view of this I face the following dilemma: Should I learn Python to be able to use its extensive scientific libraries (Pandas, Numpy, Scipy, ...), or should I just dive into similar packages of a language I'm already acquainted with (Racket, or to a lesser extent Scala)?

(Ideally I would learn Python in parallel with using statistical libraries in Racket, but I'm not sure I'll have time for both)

I'm not looking for an answer to this dilemma, but rather for feedback on my different considerations:

My current position is as follows:

In favour of Python:

  • Extensively used libraries
  • Widely used (may be decisive in case of collaboration with others)
  • A lot of online material to start learning it
  • Conferences that are specifically dedicated to Scientific Computing with Python
  • Learning Python won't be a waste of time anyway

In favour of a language I already know:

  • It's a way to deepen my knowledge of one language rather than getting superficial knowledge of one more language (under the motto: you should at least know one language really well)
  • It is feasible. Both Racket and Scala have good mathematics and statistics libraries
  • I can start right away with learning what I need to know rather than first having to learn the basics

Two concrete questions:

  1. What am I forgetting?
  2. How big of a nuisance could the Python 2 vs 3 issue be?
Improve this question
$\endgroup$
2
  • 9
    $\begingroup$ All these questions have been beaten to death on StackOverflow. What value is there in rehashing it here? $\endgroup$
    – Dirk Eddelbuettel
    Jun 16, 2014 at 14:21
  • $\begingroup$ @DirkEddelbuettel Because they are off-topic on SO, hence often closed? $\endgroup$
    – Franck Dernoncourt
    Jun 16, 2014 at 18:56

3 Answers 3

Reset to default
13
$\begingroup$

Personally going to make a strong argument in favor of Python here. There are a large number of reasons for this, but I'm going to build on some of the points that other people have mentioned here:

  1. Picking a single language: It's definitely possible to mix and match languages, picking d3 for your visualization needs, FORTRAN for your fast matrix multiplies, and python for all of your networking and scripting. You can do this down the line, but keeping your stack as simple as possible is a good move, especially early on.
  2. Picking something bigger than you: You never want to be pushing up against the barriers of the language you want to use. This is a huge issue when it comes to languages like Julia and FORTRAN, which simply don't offer the full functionality of languages like python or R.
  3. Pick Community: The one most difficult thing to find in any language is community. Python is the clear winner here. If you get stuck, you ask something on SO, and someone will answer in a matter of minutes, which is simply not the case for most other languages. If you're learning something in a vacuum you will simply learn much slower.

In terms of the minus points, I might actually push back on them.

Deepening your knowledge of one language is a decent idea, but knowing only one language, without having practice generalizing that knowledge to other languages is a good way to shoot yourself in the foot. I have changed my entire favored development stack three time over as many years, moving from MATLAB to Java to haskell to python. Learning to transfer your knowledge to another language is far more valuable than just knowing one.

As far as feasibility, this is something you're going to see again and again in any programming career. Turing completeness means you could technically do everything with HTML4 and CSS3, but you want to pick the right tool for the job. If you see the ideal tool and decide to leave it by the roadside you're going to find yourself slowed down wishing you had some of the tools you left behind.

A great example of that last point is trying to deploy R code. 'R''s networking capabilities are hugely lacking compared to python, and if you want to deploy a service, or use slightly off-the-beaten path packages, the fact that pip has an order of magnitude more packages than CRAN is a huge help.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ R is unbeatable in terms of number of stats packages. So if you go for python you still need to learn enough R to be able to use rpy2 to call the missing packages. $\endgroup$
    – Simd
    Jun 17, 2014 at 8:07
  • 1
    $\begingroup$ @Lembik I would disagree with both of those points actually. As someone who has very frequently used both R and python, I've never had the need to use rpy2, and even though R wins in pure number of packages, python has decidedly more functionality across data science. $\endgroup$
    – indico
    Jun 17, 2014 at 14:58
  • $\begingroup$ I think that may be right for data science in general although I would argue that for cutting edge stats you still need to use R packages. $\endgroup$
    – Simd
    Jun 17, 2014 at 15:27
  • 3
    $\begingroup$ @indico then you must be using the definition of "Data Science" that excludes statistics. $\endgroup$
    – Gavin Simpson
    Jun 17, 2014 at 16:15
  • $\begingroup$ I've accepted this answer since it gives the most direct answer to my question; not because it is the one true answer (read: check the other answers as well) $\endgroup$
    – Patrick Allo
    Jun 17, 2014 at 17:30
5
$\begingroup$

From my experience, the points to keep in mind when considering a data analysis platform are:

  1. Can it handle the size of the data that I need? If your data sets fit in memory, there's usually no big trouble, although AFAIK Python is somewhat more memory-efficient than R. If you need to handle larger-than-memory data sets, the platform need to handle it conveniently. In this case, SQL would cover for basic statistics, Python + Apache Spark is another option.
  2. Does the platform covers all of my analysis needs? The greatest annoyance I've encountered in data mining projects is having to juggle between several tools, because tool A handles web connections well, tool B does the statistics and tool C renders nice pictures. You want your weapon-of-choice to cover as many aspects of your projects as possible. When considering this issue, Python is very comprehensive, but R has a lot of build-in statistical tests ready-to-use, if that's what you need.
Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Your point 2 is very important. Juggling with different tools means importing data. And data importing is an enormously error prone step and feels extremely unproductive - so avoid it, whereever you can. $\endgroup$
    – Christian Sauer
    Jun 16, 2014 at 13:25
1
$\begingroup$

According to me, all the factors, you have mentioned are superficial in nature. You have not considered the core of tool selection. In this case, there are 2 aspects, you mentioned:

  1. Data analysis - What kind of analysis are you working on? There might be some analysis which are easier in some languages and more difficult in other.

  2. Visualization - R provides similar community and learning material (as Python) and has the best visualizations compared to other languages here.

At this stage, you can be flexible with what language to learn, since you are starting from scratch.

Hope this helps.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I agree that I'm still only scratching the surface. That's mainly the result of being in the early stages of my project-design. (I had initially ruled out R since I was more interested in using a general purpose language) $\endgroup$
    – Patrick Allo
    Jun 16, 2014 at 9:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.