Hadoop NameNode failed to start, Error: FSNamesystem initialization failed. java.io.FileNotFoundException

Exception I am getting is,

2011-07-13 12:04:13,006 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.FileNotFoundException: File does not exist: /opt/data/tmp/mapred/system/job_201107041958_0120/j^@^@^@^@^@^@
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedSetPermission(FSDirectory.java:544)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLog.loadFSEdits(FSEditLog.java:724)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:992)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:812)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:364)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)

2011-07-13 14:45:02,780 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.FileNotFoundException: File does not exist: /opt/data/tmp/mapred/system/job_201107041958_0120/j^@^@^@^@^@^@
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedSetPermission(FSDirectory.java:544)
        at org.apache.hadoop.hdfs.server.namenode.FSEditLog.loadFSEdits(FSEditLog.java:724)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:992)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:812)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:364)
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
        at org.apache.hadoop.h开发者_开发知识库dfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)

---

Problem(editlogs got corrupted) got resolved. I used -importCheckpoint option.

I just would like to tell you the possible scenario/reason of editlog corruption might have happened(correct me if I am wrong),

Below were the typical configurations in hdfs-site.xml

hadoop.tmp.dir : /opt/data/tmp
dfs.name.dir : /opt/data/name
dfs.data.dir : /opt/data/data
mapred.local.dir : ${hadoop.tmp.dir}/mapred/local

/opt/data is an mounted storage, size is 50GB. Namenode, SecondaryNamenode( ${hadoop.tmp.dir}/dfs/namesecondary) & Datanode directories were configured within /opt/data itself.

Once I moved 3.6GB compressed(bz2) file, I guess /opt/data memory usage of this dir. could have been 100%(I checked($df -h) after this incident). Then, I ran Hive with simple "Select" query, its job.jar files also needs to be created within the same directory which already has no space. So this is how the editlog corruption could have been occurred.

This is really a good learning for me! Now I have changed that configurations.