If I understand the Databricks philosophy correctly, Spark will soon be heavily moving toward dataframes, i.e. away from the usual map/reduce on RDDs. I was wondering if there are any good suggestions for online courses or books that introduce Spark from the dataframe point of view? I know Databricks has a good number of resources on dataframes but I would really like to see some more detailed courses.
edX offers some spark courses. Try the following:
I believe this is through a partnership with Databricks
Alex, you can check in this Coursera Big Data Course a chapter about Spark DataFrames in "Week 5", it's an introduction but it explains how to setup PySpark for DataFrames, may be for your situation it would be a good start.