I use python in my day to day work as a research scientist and I am interested in learning C. When would a situation arise where python would prove insufficient to manipulate data?


Most of the common libraries you would use for data manipulation do actually use C (or C++ or Fortran, etc.) under the hood.

There are even libraries such as CuPy, which offers the entire NumPy API, but can run your code on a GPU. Using GPUs for speed is a much more common use case these days (in my experience), compared to writing the C/C++ version.

EDIT: here is a related answer, about which programming languages are most competitive for AI, ML, DataScience, etc.

In my opinion, you might need to "do it yourself" in one of 3 cases:

1. Speed

You need it to run faster than current libraries offer - e.g. if the clustering algorithms in Scikit-Learn are too slow

2. Memory

You need to use less memory that existing algorithms - perhaps a specific method on your Pandas DataFrame uses more memory that you have available

3. New Algorithms

You need something that is fairly fast or very low level, and no existing library offers it. I would normally suggest trying your idea first using NumPy though, before trying to roll your own binaries.

  • $\begingroup$ Thank you for your answer. I'm curious about Julia as well, I've heard that it runs as fast as C but the syntax to me looks much like python and seems like it would be a good skill to acquire. I know it's not widely used, but it looks like its almost meant to be a "python killer" in the sense that it is designed to easily allow people to migrate from python to Julia while offering substantial performance boosts? Is Julia significantly more powerful than for instance pandas? $\endgroup$ – Angus Campbell Jan 28 at 20:24
  • $\begingroup$ Julia has its pros and con. I added a link to another questions, where I share my opinion on the most practical combination of languages. Overall, I think you are best sticking with Python, and you will be able to decide for yourself which you need to learn, once you know which problems you want to solve :) $\endgroup$ – n1k31t4 Jan 28 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.