Posting details from the link in comments above.
The data can be partitioned among multiple processors (or streams or threads). Each processor can read the previous iteration's cluster centers and assign the rows on the processor to clusters. Each processor then calculates new centers for its of data. Each actual cluster center (for the data across all processors) is then the weighted average of the centers on each processor.
In other words, the rows of data do not need to be combined globally.
They can be combined locally, with the reduced set of results combined
across all processors. In fact, MapReduce even contains a "combine"
method for just this type of algorithm.
You postulate a shared file that has the centroids as calculated for
each processor. This file contains:
An iteration through the algorithm is going to add another set of
rows to this file. This information is the only information that
needs to be communicated globally.
There are two ways to do this in the MapReduce framework. The first
uses map, combine, and reduce. The second only uses map and reduce.
link with all details