Problem 26: Equivalence of Two MapReduce Models

From Open Problems in Sublinear Algorithms
Jump to: navigation, search
Suggested by Paul Beame
Source Kanpur 2009
Short link https://sublinear.info/26

The original MapReduce paper [DeanG-04] gives two distributed models. First it only says that intermediate key/value pairs with the same key are combined and sent as batch jobs to workers. Then in Section 4.2, it additionally guarantees that the batch jobs received by a single worker are sorted according to the corresponding key values. There are algorithms that rely on this additional feature of MapReduce. Are these two models equivalent? For decision problems in the complexity world, we know strong time-space trade-offs for sorting, but no similar lower bounds are known for distinctness.