Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Apache Hadoop was once considered one of the tools you should have as a Data Scientist. Around 2012 to 2014, it gained the popularity steeply among Data Scientists, and seemed to be considered one of the toolsets on the same boat as Python, R, and SQL.
But these days I have not heard its story. In fact, the Google Trend also agrees with me, that the popularity is peak at May, 2015.
Why has Hadoop failed to become one of the necessary tools as a Data Scientist, like Python or R?
I wouldn't say Hadoop failed to become popular rather I would say, it is still the base of any production big data system.
Python or R are handy at the beginning when you just need to try out things but when it comes to putting things in production, Hadoop is the way to go. It does not directly provide any data scientist tools what it does provide is the base where data would be stored, processed and can be used to apply machine learning algorithms using Spark.
So to summarize, I see Hadoop being used in the following contexts
Hope this helps.
You are comparing apples with oranges. Hadoop is one of the backend of big data platform and Python/R are programming languages which are used built prediction models and data pipeline. Hadoop still can be used as data storage however more robust and faster distributed data storage frameworks are gaining popularity such as Apache Spark hence Hadoop lost its charm.
