Apache Spark is an open source cluster computing system that aims to make data analytics fast — both fast to run and fast to write, originally developed in the AMPLab at UC Berkeley.

From http://spark.apache.org/:

Apache Spark is an open source cluster computing system that aims to make data analytics fast — both fast to run and fast to write.

To run programs faster, Spark offers a general execution model that can optimize arbitrary operator graphs, and supports in-memory computing, which lets it query data faster than disk-based engines like .

Spark is not tied to the two-stage paradigm, and promises performance up to 100 times faster than Hadoop MapReduce

Spark provides primitives for in-memory cluster computing that allows user programs to load data into a cluster's memory and query it repeatedly, making it well suited to machine learning algorithms. To make programming faster, Spark provides clean, concise APIs in , and . You can also use Spark interactively from the and shells to rapidly query big datasets.

Spark runs on , , standalone, or in the cloud. It can access diverse data sources including , , , and .

Recommended reference sources:

Spark Documentation

Learning Spark - Lightning-Fast Big Data Analysis

AMP Camp 5 (Berkeley, CA, November 20-21, 2014)

AMP Camp 4 (Strata Santa Clara, Feb 2014) — focus on BlinkDB, MLlib, GraphX, Tachyon

AMP Camp 3 (Berkeley, CA, Aug 2013)

AMP Camp 2 (Strata Santa Clara, Feb 2013)

AMP Camp 1 (Berkeley, CA, Aug 2012)

history | excerpt history