I'm new to apache spark.
Is it possible to configure multi cluster spark without hadoop?
If so, can you please provide the steps. I would like to create clusters on Google Compute Engine (1-master, 1-worker)
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this communityI'm new to apache spark.
Is it possible to configure multi cluster spark without hadoop?
If so, can you please provide the steps. I would like to create clusters on Google Compute Engine (1-master, 1-worker)
As explained in the comments by @Emre, there is a python script on github which you can use to set up a cluster of whatever size you like: Spark Cluster on Google Compute Engine | Ido Green. It allows you to specify the instance type to use for the workers (slaves) and separately what to use for the master.
Note that it uses gcutil, which is deprecated in favor of gcloud compute