I'm a data analyst/scientist working mostly with the Python open source stack i.e. Pandas, scikit-learn, matplotlib, seaborn etc. I want to expand my toolbox and learn a distributed computing framework.
Hadoop had created a lot of fuss in around 2013-2014. As per my limited knowledge on the subject, Apache Spark has improved upon Hadoop multi-fold in all aspects.
So, apart from the obvious case when you need to maintain legacy Hadoop applications, is there any reason as of today to prefer Hadoop over Apache Spark?