Feature Proposal: Add message framing to binary stream i/o about websocat HOT 8 CLOSED

Do you mean something like a binary (more performant) analogue of Websocat's --base64 --base64-text --binary-prefix --text-prefix (i.e. a mode which allows to mix and match arbitrary text and binary messages with preserved boundaries and whitespace)?

from websocat.

In that direction, yes. The existing binary versions separate message on newline or nul char but all bytes are valid in my binary messages.

from websocat.

To add, --binary-prefix is not usable for larger messages that are greater than read buffer size and cannot be "replayed" (stored as a file and redirected to stdin) to another application. I may have to use a rather long prefix which is not performant.

from websocat.

You can adjust buffer size with -B (unless you want the whole big file as one big WebSocket message).

Currently WebSocat assumes any WebSocket message can fit in memory (maybe multiple times).

from websocat.

What other websocat features do you use?

If it's just some_program | websocat -b ws://... then maybe it would be better to create a dedicated, simpler tool for it?

(though working with larger-than-RAM WebSocket messages would require to drop to low level and work with WebSocket frames instead)

from websocat.

Implemented lengthprefixed: overlay - it allows 1 to 8 of big- or little-endian length bytes to denote message boundaries and works in both directions.

Is it what you need? Documentation snippet below:

lengthprefixed:

Internal name for --dump-spec: LengthPrefixed

Turn stream of bytes to/from data packets with length-prefixed framing. [A]

You can choose the number of header bytes (1 to 8) and endianness. Default is 4 bytes big endian.

This affects both reading and writing - attach this overlay to stream specifier to turn it into a packet-orineted specifier.

Mind the buffer size (-B). All packets should fit in there.

Examples:

websocat -u -b udp-l:127.0.0.1:1234 lengthprefixed:writefile:test.dat

websocat -u -b lengthprefixed:readfile:test.dat udp:127.0.0.1:1235

This would save incoming UDP packets to a file, then replay the datagrams back to UDP socket

websocat -b lengthprefixed:- ws://127.0.0.1:1234/ --binary-prefix=B --text-prefix=T

This allows to mix and match text and binary WebSocket messages to and from stdio without the base64 overhead.

from websocat.

Thank you, this looks great.

from websocat.

Closing, thank you!

from websocat.