Having looked at various job requirements, I see that a lot of them require you to be an expert in Hadoop, use Spark but I am wondering how useful are these technologies (due to my lack of understanding)?
It seems that whatever I need to do, I always am able to solve using Google Cloud. For example, I can train models on huge datasets (I've only tried GBs so far) using Cloud ML. I can use BQ/Redshift for data prep and so on.
So my question is, how future proof are Spark/Hadoop, why are they an important skill to learn in 2018 and will they be replaced/getting replaced already?