honeytail exits without reading from named pipes as input source about honeytail HOT 3 CLOSED

This is the collector side via strace.

fstat(3, {st_mode=S_IFIFO|0644, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe3f89bb000 lseek(3, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) write(3, "{\"sys\":{\"hostname\":\"lib-admin1.l"..., 159) = -1 EPIPE (Broken pipe) --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=5946, si_uid=0} --- +++ killed by SIGPIPE +++

When I start the collector first it sits waiting for honeytail to open the FIFO. When honeytail starts up, the collector dies from a SIGPIPE once honeytail closes the FIFO and exits. When I start honeytail first, it pauses waiting for the collector to open the FIFO. When it does, this is honeytail:

select(0, NULL, NULL, NULL, {0, 3}INFO[0000] Starting honeytail ) = 0 (Timeout) write(3, "\25\3\3\0\32\0\0\0\0\0\0\0\2\370\5\362W\276b\214\fa\310T\2718\336\354\22\346\220", 31) = 31 epoll_ctl(4, EPOLL_CTL_DEL, 3, {0, {u32=0, u64=0}}) = 0 close(3) = 0 epoll_wait(4, {}, 128, 0) = 0 select(0, NULL, NULL, NULL, {0, 3}) = 0 (Timeout) futex(0xc4202e4910, FUTEX_WAKE, 1) = 1 futex(0xa784f0, FUTEX_WAIT, 0, NULLDEBU[0000] getStartLocation seeking to offset in logfile starting at=0 DEBU[0000] about to call tail.TailFile conf={Paths:[/var/run/syspoll/fifo] Type:0 Options:{ReadFrom:last Stop:false Poll:false StateFile:}} location=&{Offset:0 Whence:0} statefile=/tmp/fifo.leash.state tailConf={Location:0xc420498310 ReOpen:true MustExist:true Poll:false Pipe:false RateLimiter:<nil> Follow:true MaxLineSize:0 Logger:0xc42008ebe0} DEBU[0006] Initializing stats reporting. Will print stats once/60 seconds ) = 0 select(0, NULL, NULL, NULL, {0, 3}) = 0 (Timeout) write(2, "\33[37mDEBU\33[0m[0006] lines channe"..., 68DEBU[0006] lines channel is closed, ending json processor ) = 68 futex(0xa784f0, FUTEX_WAIT, 0, NULLINFO[0006] Summary of sent events avg_duration=0s count=0 count_per_status=map[] errors=map[] fastest=0s lifetime_count=0 response_bodies=map[] slowest=0s INFO[0006] Total number of events sent number sent by response status code=map[] total attempted sends=0 INFO[0006] Honeytail is all done, goodbye! <unfinished ...> +++ exited with 0 +++

from honeytail.

I ran into this too. As another workaround that doesn't require changing the producer, one can use tail to feed the named pipe into honeytail's stdin. For an upstart service, this exec line seems to work:

exec su -s /bin/bash honeycomb -c 'set -eo pipefail; tail -F -c +0 /path/to/fifo | /usr/bin/honeytail --config=/etc/honeytail/honeytail.conf --file=-'

from honeytail.

Hello,

As we enter 2022 Q1, we are trimming our OSS backlog. We will be closing this issue as it is a low priority for us to fix. It is unlikely that we'll ever get to it, and so we'd like to set expectations accordingly.

If this issue is important to you, please feel free to ping here and we can discuss/re-open.

from honeytail.