Hadoop in 24 Hours, Sams Teach Yourself by Jeffrey Aven

Hadoop in 24 Hours, Sams Teach Yourself by Jeffrey Aven

Author:Jeffrey Aven [Jeffrey Aven]
Language: eng
Format: epub
Publisher: SAMS
Published: 2017-04-13T16:00:00+00:00


Figure 13.5 Loading an RDD From a Text File in a Distributed Filesystem.

RDDs can also be loaded from other data sources such as relational databases using a JDBC interface, as well as by simply parallelizing collections of objects such as data structures in Python or Scala.

RDD Persistence and Re-use

RDDs are created and exist predominantly in memory on executors. By default, RDDs are transient objects that exist only while they are required. Once they are transformed into new RDDs and no longer needed for any other operations, they are removed permanently. This may be problematic if an RDD is required for more than one action because it must be re-evaluated in its entirety each time. To address this issue, Spark provides API methods to persist, cache and checkpoint RDDs.



Download



Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.