reduce logging when exiting writeLoop

[tj]

investigating the broken pipe issue but the logs quickly fill up with thousands of these lines while it slowly waits for the remaining work. Is it possible to lower the log interval? Maybe just tick every second and/or when the in-flight is reduced

red-files 2014/10/30 15:52:10 ERR    1 [red-files/red-extract] (:5001) IO error - write tcp 127.0.0.1:5001: broken pipe
red-files 2014/10/30 15:52:10 ERR    1 [red-files/red-extract] (:5001) error sending command FIN 07526046660c5002 - write tcp 127.0.0.1:5001: broken pipe
red-files 2014/10/30 15:52:10 INF    1 [red-files/red-extract] (:5001) beginning close
red-files 2014/10/30 15:52:10 INF    1 [red-files/red-extract] (:5001) breaking out of writeLoop
red-files 2014/10/30 15:52:10 INF    1 [red-files/red-extract] (:5001) writeLoop exiting
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
2014/10/30 15:52:11 stats:
2014/10/30 15:52:11   events.published 2419.11/s tick=24,192 total=985,360
2014/10/30 15:52:11   files.processed 0.20/s tick=2 total=2,142
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 31 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 30 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) delaying close, 30 outstanding messages
red-files 2014/10/30 15:52:11 INF    1 [red-files/red-extract] (:5001) readLoop exiting
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 30 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 30 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:11 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:12 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:13 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight
red-files 2014/10/30 15:52:13 WRN    1 [red-files/red-extract] (:5001) draining... waiting for 29 messages in flight

[mreiferson]

absolutely, I noticed this as well - I've actually already started working on this in #89 so hang tight and we'll get this cleaned up.

Thanks @tj

[mreiferson]

see #89

[tj]

rad!

[gm42]

The 1 second limit set in that PR is not enough..I found today 1.9GB of the same log entry, over the arch of 12 hours.

This happens when the consumer is starving so perhaps in that case log entries could be reduced.