MapReduce max marks per college

We use one MapReduce job. The mapper emits (college, marks) for each record. The framework groups values by key (shuffle and sort). The reducer takes the maximum of the list of marks for each college.

Mapper and reducer pseudocode

map(key = byte offset, value = "Name,College,Marks"):
    fields = split(value, ",")
    emit(trim(fields[1]), int(trim(fields[2])))   // (college, marks)

reduce(key = college, values = list of marks):
    max = 0
    for v in values:
        if v > max: max = v
    emit(college, max)

Input to the map function

Each row is one (key, value) pair. The key is the byte offset of the line (not used), the value is the line.

(Offsets are shown as line numbers for simplicity.)

Output of the map function (input to shuffle)

(ABC, 70)   (ABC, 80)   (ABC, 60)
(XYZ, 90)   (XYZ, 80)
(PQR, 90)   (PQR, 70)   (PQR, 60)

After shuffle and sort (input to the reduce function)

The framework groups all values of the same key and sorts keys:

Output of the reduce function (final output)

Data flow

 Input lines -> MAP -> (college,marks)
      -> SHUFFLE/SORT -> (college,[marks])
      -> REDUCE -> (college,max)

Since max is associative and commutative, the same reducer can also be used as a combiner, to reduce network traffic by sending only the local maximum from each mapper.