I am executing the job as:
hadoop/bin/./hadoop jar \ /home/hadoopuser/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar \ -D mapred.reduce.tasks=2 \ -file kmeans_mapper.py \ -mapper kmeans_mapper.py -file kmeans_reducer.py \ -reducer kmeans_reducer.py \ -input gutenberg/small_train.csv \ -output gutenberg/out
When the two reducers are done, I would like to do something with their output. The MapReduce tutorial states:
Users may need to chain MapReduce jobs to accomplish complex tasks which cannot be done via a single MapReduce job. This is fairly easy since the output of the job typically goes to distributed file-system, and the output, in turn, can be used as the input for the next job.
but it does not give any example. Can I make this happen?
Relevant question of mine, but I decided to ask the experts too! :)
A negative answer would still be very useful!