I am executing the job as:

hadoop/bin/./hadoop jar                                \
/home/hadoopuser/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar                                                  \
-D mapred.reduce.tasks=2                               \
-file kmeans_mapper.py                                 \
-mapper kmeans_mapper.py -file kmeans_reducer.py       \
-reducer kmeans_reducer.py                             \
-input gutenberg/small_train.csv                       \
-output gutenberg/out

When the two reducers are done, I would like to do something with their output. The MapReduce tutorial states:

Users may need to chain MapReduce jobs to accomplish complex tasks which cannot be done via a single MapReduce job. This is fairly easy since the output of the job typically goes to distributed file-system, and the output, in turn, can be used as the input for the next job.

but it does not give any example. Can I make this happen?

Relevant question of mine, but I decided to ask the experts too! :)

A negative answer would still be very useful!


I can safely conclude you cannot.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.