Hadoop pipes (wordcount) example failing (with hadoop 0.21.0)

I was trying out the simplistic word count example for hadoop pipes. Unfortunately it is erroring out with java.lang.NullPointerException and /usr/lib64/libstdc++.so.6: no version information available errors when I try to run it as:

$ bin/hadoop pipes -Dhadoop.pipes.java.recordreader=true \
                   -Dhadoop.pipes.java.recordwriter=true \ 
                   -input <inputHDFSPath> \               
                   -output <outputHDFSPath> \
                   -program <cppBinaryHDFSPath>

The error details are here.

A开发者_如何学Gony suggestions on how to fix this would be greatly appreciated. Thanks.

Edit1: Added command used to run.

Edit2: It turns out that the C++ binary wasn't statically linked. After rectifying that I am now seeing this error.

Edit3: Added hadoop version being used. I have also added relevant portions of the jobtracker logs here. Also, stderr for each of the failed attempts has this:

$more /usr/local/dp/hadoop/userlogs/job_201107211831_0013/\
attempt_201107211831_0013_m_‌​000000_0/stderr 
Hadoop Pipes Exception: Aborted by driver

---

since it is looking for /usr/lib64/libstdc++.so.6 it seems you mapper is compiled with 64bit compiler, are you sure the jvm is also 64bit?

I have a feeling it is this 64 bit thing which is causing the broken pipes. check on the cluster and the dev machine.

---

So I gave it another shot and started from scratch. This time round I had better success. Here is the code and the config files I used.

Also, here's the command I used to run the job:

hadoop pipes -conf path/to/dp_hpt_conf.xml 
             -input /path/to/input 
             -output /path/to/output

The input files in the /path/to/input were:

$hadoop fs -ls /path/to/input
Found 3 items
-rw-r--r--   3 dp dpgrp         25 2011-07-26 23:24 /path/to/input/bar
-rw-r--r--   3 dp dpgrp         87 2011-07-26 23:24 /path/to/input/baz
-rw-r--r--   3 dp dpgrp         44 2011-07-26 23:24 /path/to/input/foo
$hadoop fs -get /path/to/input/foo .
$cat foo 
The quick brown fox jumps over the lazy dog

And the output was:

$ hadoop fs -get /path/to/output/part-00000 .
$ cat part-00000 
another 3
be      2
brown   2
dog     2
end     2
for     2
fox     2
input   2
is      3
jumps   2
lazy    2
of      2
over    2
program 2
quick   2
sentence        3
should  2
test    2
that    2
the     5
this    4
yet     2

(Apologies for the wonky paths. My actual paths were different - the paths here are just my lazy attempts at anonymizing them.)