开发者

How to write a custom flume OG sink

开发者 https://www.devze.com 2023-03-17 14:57 出处:网络
We\'re using flume and I need to collect some log messages into rabbitmq. I found a source implementation that reads messages from rabbitmq, but I couldn\'t find a sink that can write message开发者_JA

We're using flume and I need to collect some log messages into rabbitmq. I found a source implementation that reads messages from rabbitmq, but I couldn't find a sink that can write message开发者_JAVA百科s into rabbit. So I was thinking about writing one myself. Looking at sample implementations like logsandra made me think it shouldn't be too difficult.

However I couldn't find any documentation on how to write a custom sink. I didn't find a maven repo for the flume jars, or setup instructions on how to deploy a custom sink.

Can anyone share his experience, or better, point me to an existing tutorial.


Eventually we wrote one ourselves, and open sourced it - https://github.com/kenshoo/flume-rabbitmq-sink

there's a cloudera repo now https://ccp.cloudera.com/display/CDHDOC/Using+the+CDH3+Maven+Repository (there wasn't when I asked)

And flume user guide now explains well how to configure a custom plugin

http://archive.cloudera.com/cdh/3/flume/UserGuide/index.html#_extending_via_sink_source_decorator_plugins

0

精彩评论

暂无评论...
验证码 换一张
取 消