Is it possible to have a buffer of bytes? about bindata HOT 7 CLOSED

Is the data structured or arbitrary? BinData's goal is to declaratively define parsers for structured data.

If your data is structured, then see Choice.

I'm unclear as to what you want.

from bindata.

Sorry for the confusion. I'm building a Ruby application that will read data from a JSON API, serialize it into binary format and send it to a device. The device speaks binary, it does not handle JSON, so I'm locked into that.

The problem is seen on the device, which uses a buffer of bytes for all data, since the actual data received can be a string, an integer or a float.

Good example is sending back a packet with a response code. The code is an integer, however, due to generic nature of the payload, I have been using a string. So when the payload is encoded, instead of it being represented as a byte (\0 for example), it's represented as a string 0 (0x30 in ASCII). This makes sense, since I'm using string data type.

I'm wondering if it's possible to create a string-like object that serializes the data into a byte stream. So that when a value is stored, it's automatically converted into it's binary representation. The onus is on the consumer to properly parse the data, the stream just automatically converts an integer, a character / string, a float, UTF, etc. into it's binary representation.

from bindata.

It's not clear (to me) the specifics of what you are trying to achieve. Could you perhaps provide an example of what you would like, e.g. the JSON you read and the corresponding binary format that should be emitted? That would go a long way in making it easier for us to try to assist.

from bindata.

Have you heard of http://bsonspec.org/ Binary JSON? Why reinvent the wheel.

from bindata.

@acds yes, but that means I have to use JSON on the device and that's not an option right now.

The problem I'm trying to solve is not serializing JSON, the problem is converting JSON to a custom binary format.

@aidansteele here's an example

Example JSON:

{
  type: 0,
  message: "OK"
}

This is converted into internal message stored ok OK as 0. The data structure to store it is

class MyData < BinData::Record
  uint8    type
  uint32  length
  string   data
end

If you were to run to_binary_s on that message, it would output (assuming big endien):

\0\0\0\0\10

Notice the last 0, it's not reported as \0, but as 0. This means when that last 0 is put on the wire, it's transmitted as it's ASCII value (0x30), not as \0.

You can check this easily by running wireshark and capturing a packet with the above as your data.

What I'm hoping to have happen is that the binary looks like this

\0\0\0\0\1\0

This would treat the last 0 as an actual 0x0

However, if you were to have the data field store A, then the result should be:

\0\0\0\0\2\4\1

The common format, if you like, is that no matter the value, it's converted to it's binary representation.

With C, it's fairly simple, just declare a byte array, like uint8_t* array, and then use memcmp to store the value in that array. Doesn't matter what the value is, it just gets copied in byte by byte, without worrying about casting.

Hope this is a little more clear. I realize I'm asking about a very odd case, given JSONs ubiquity, unfortunately, I have to play with bits in this case.

from bindata.

I'm still don't understand your data format. Strings are encoded as bytes strings while integers are encoded as 8bit packed nibbles?

The onus is on the consumer to properly parse the data, the stream just automatically converts an integer, a character / string, a float, UTF, etc. into it's binary representation.

If this is a new binary data format then I recommend you read up on TLVs. Then have a look at some of the example code.

If you just want something like memcpy then don't use the declarative classes (Record etc) of BinData. Use the primitive classes and handle it imperatively. I don't recommend this approach though it is possible.

type = BinData::Uint8.new(0)
length = BinData::Uint32Be.new(1)
value = BinData::Uint8.new(0)
io = Socket ...
type.write(io)
length.write(io)
value.write(io)

or use Array#pack.

from bindata.

@dmendel Thanks, i'll try that. Closing the issue.

from bindata.