I have written my custom partitioner for partitioning datasets. I want to partition two datasets using the same partitioner and then in the next mapreduce job, I want each mapper to handle the same partition fr开发者_如何学Goom the two sources and perform some function such as joining etc. How I can I ensure that one mapper gets the split that corresponds to same partition from both the sources?
Any help would be highly appreciated.
What you are describing is one variation of a map-side join. Chapter 8 of Pro Hadoop or org.apache.hadoop.mapred.join
精彩评论