Chukwa is designed to tail log files, and stream delta to HDFS as soon as
the log is written.  There is artificial delay put in place of a few
milliseconds to throttle CPU usage.  We haven't done much performance
analysis, but there is probably a lot room for optimizations to make Chukwa
faster.  It will be interesting to see support for compressed logs, and
patches are welcome.


On Wed, Jan 10, 2018 at 5:18 AM, Popa Nicolae <[EMAIL PROTECTED]> wrote: