Notice that after a vector, matrix row, or matrix column has been
spread, multiple copies of the data exist. Often, as part of
an operation, multiple instances of data exist, but each instance
holds a contribution to a global result. These contributions
must then be
reduced (typically summed) into the global result.
Often, the reduction needs only be performed (simultaneously) within
one or all row(s) or
column(s) of the node mesh.
Looking ahead slightly, consider the situation in Figure 1.5.
Each column of nodes holds a column vector, which must be summed
element-wise into a single vector. The collective
communication requires is a reduction that leaves a sub-vector of
the result on each node. We call this a
distributed reduction. We will discuss reduction (consolidation)
of data again in Chapter 
.