Big Data with Hadoop MapReduce by Jeyaraj Rathinaraja; Paul Anand; Pugalendhi Ganeshkumar

Author:Jeyaraj, Rathinaraja; Paul, Anand; Pugalendhi, Ganeshkumar
Language: eng
Format: epub
Publisher: Apple Academic Press, Incorporated
Published: 2021-08-15T00:00:00+00:00

Reduce slow start

Reduce function is executed only after the inputs from all map tasks are received. However, the map output collection process (reduce phase) is started only after a certain number of map tasks are completed. Schedulers wait until 5% of map tasks get over before reduce phase starts. For large jobs, this can cause problems with cluster utilization because they hold reduce containers while waiting for the map tasks to complete. Setting mapreduce. job.reduce. slowstart.completedmaps to a higher value, 0.8 (80%), can help improve throughput.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.