Learning Spark Summary: Lightning-Fast Big Data Analytics - Developer Deconstructed by McGrath Todd

Author:McGrath, Todd [McGrath, Todd]
Language: eng
Format: azw3
Tags: Computers & Technology
Published: 2015-12-15T16:00:00+00:00

6) Driver sends work (tasks) to executors

7) Tasks are run on executor processes

8) If the driver’s main() method exits or it calls SparkContext.stop(), it will terminate the executors and release resources from the cluster manager.

Examples of packaging a Spark application in Python with py-files argument, Scala through sbt and Java via maven are provided.

Amazon EC2

Spark download provides a script to launch clusters on Amazon EC2. This script launches a set of EC2 nodes and installs the Standalone cluster manager on them. The script is called spark-ec2

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.