Broken segments about orcanode HOT 4 OPEN

Last night it appears Sunset Bay restarted successfully, but streamed .ts segments with only zero-amplitude audio data. It feels like this happens once in a blue moon, but in this case I logged in with Dataplicity and confirmed with htop that both ffmpeg and upload_s3.py were running. Also, arecord -l suggested that the audio hardware device was pisound as expected.

From the S3 and player perspectives, the .m3u8 manifest and .ts segments seemed to be in place, but the silent .ts segments in the streaming timestamped bucket were anomalously small -- ~22 kB instead of the typical 107 kB file size for that location, e.g. in the next bucket timestamp that seemed to resolve the issue (via a manual stop/start of the container).

Here is an example clip that a listener tagged as silent -- https://live.orcasound.net/reports/cand_02v0gyoe1eO4nw9uOw6Sjt

Here is screenshot to the player before/after the container restart:

And here is a view of the zero-amplitude data in Audacity (after download of .ts segments and transcoding to mp3 via mp4...):

from orcanode.

Possibly related issue -- https://www.reddit.com/r/ffmpeg/comments/manakd/no_audio_on_hls_stream/

from orcanode.

A first this morning reviewing in Audacity the archived lossy .ts segments from this morning (6:30am local) concatenated into a continuous .mp3 file by my ts2mp3.sh script: I discerned a data gap 10-seconds into the recording. I noticed this because the first 10 seconds happened to have low ambient noise; then there is a jump to boat noise that clearly indicates a gap of many seconds or minutes.

This suggests a pretty major bug in the streaming code or upload performance from the Orcasound Lab hydrophone. Is this sort of gap (part of) the cause of the ~10-minutes of missing data per ~6-hour data streaming container run?

Logging in remotely with Dataplicity, I see these hints at a (pretty old) version of the orcanode code:

from orcanode.

A final note: in further reviewing the last ~18 hours of streamed data from Orcasound Lab to understand when J pod first reached Haro Strait from points north, I discovered a full 6-hour data segment missing from the S3 bucket (5/4/24 18:30 - 5/5/24 00:30) --

-- likely related to (and now noted within) issue #18

from orcanode.