Hadoop MapReduce 1.1.1 Chaining Example
I am a very beginner in Hadoop, but in Hadoop 1.1.1 I am trying to make map -> shrink -> map -> shrink. I'm having a hard time finding a concrete example of how to do this in Java. I'm really wondering what the whole driver method looks like. I am currently using libraries org.apache.hadoop.mapreduce
(which I believe are the newest codebase).
Also, I would rather not install another library like Oozie if I don't need to.
source to share
I think this might help you: http://blogs.msdn.com/b/avkashchauhan/archive/2012/03/29/how-to-chain-multiple-mapreduce-jobs-in-hadoop.aspx
You can also take a look at ChainMapper ( http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapred/lib/ChainMapper.html ) depending on what you are trying to achieve. However, this only works with one reduction. If you want to go to map -> shrink -> map -> shrink, I would go with my first link.
source to share