O Task Granularity Fine granularity tasks: map tasks > machines Minimizes time for fault recovery Can pipeline shuffling with map execution Better dynamic load balancing Often use 200, 000 map 5000 reduce tasks Running on 2000 machines Process Time ---------------------- User Program Map Reduce waIt Master Assign tasks to worker machines Worker I Map 1 Map 3 Worker 2 Map 2 Worker 3 Read I1 Read 1.3 Read 1.2 Reduce l Worker 4 Read 2.1 Read 2.2 Read 2.3 Reduce 2Task Granularity 17 • Fine granularity tasks: map tasks >> machines – Minimizes time for fault recovery – Can pipeline shuffling with map execution – Better dynamic load balancing • Often use 200,000 map & 5000 reduce tasks • Running on 2000 machines