开发者

Programmatic equivalent of 'hadoop fs -tail -f'

开发者 https://www.devze.com 2023-03-24 02:25 出处:网络
I want to tail an hdfs file programmatically using the org.apache.hadoop.fs.FileSystem API. Is there a way to tail the file using the API in a way which is equivalent to hadoop fs开发者_JS百科 -tail -

I want to tail an hdfs file programmatically using the org.apache.hadoop.fs.FileSystem API. Is there a way to tail the file using the API in a way which is equivalent to hadoop fs开发者_JS百科 -tail -f command?


Maybe I misunderstand the question. hadoop fs -tail -f is implemented using the API isn't it?

From org.apache.hadoop.fs.FsShell.tail(String[], int)

long fileSize = srcFs.getFileStatus(path).getLen();
long offset = (fileSize > 1024) ? fileSize - 1024: 0;

while (true) {
  FSDataInputStream in = srcFs.open(path);
  in.seek(offset);
  IOUtils.copyBytes(in, System.out, 1024, false);
  offset = in.getPos();
  in.close();
  if (!foption) {
    break;
  }
  fileSize = srcFs.getFileStatus(path).getLen();
  offset = (fileSize > offset) ? offset: fileSize;
  try {
    Thread.sleep(5000);
  } catch (InterruptedException e) {
    break;
  }
}
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号