I am trying to create a mapper only job via AWS (a streaming job). The reducer field is required, so I am giving a dummy executable, and adding -jobconf mapred.map.tasks=0 to the Extra Args box. In the hadoop environment (v开发者_StackOverflow中文版ersion 0.20) I've installed, no reducer jobs will launch, but in AWS the dummy executable launches and fails.
How can run a no reducer/ mapper only jobs in AWS?
you can also use cat or NONE as reducer argument.
There is no way to create a mapper only job in amazon M/R. I am using the Identity reducer (org.apache.hadoop.mapred.lib.IdentityReducer) instead.
精彩评论