Giter Club home page Giter Club logo

node-protobuf-lite's Introduction

protobuf-lite

WORK IN PROGRESS

Light-weight protocol buffers implementation. Reads and writes the wire protocol and converts some of the more common data types. The program data is kept in native JavaScript format, so no true 64-bit ints. Also no maps, services, groups, extensions, no .proto file parsing.

On the other hand, it is possible to decode protobuf numbers and strings with a single call, eg

// pack as varint, uint32, string, float
var buf = pbuflite.pack('iVaf', [123, 4.5, 'hello', 1234.5 ]);
var items = pbuflite.unpack('iVaf', buf);
// => [ 123, 4, 'hello', 1234.5 ]

NOTE This is a work in progress. The code seems to work, but testing has been very light. Not much error checking yet internally.

Api

pack( format, array )

Encode the data in the array accorting to the format, and return a Buffer containing the packed bytes. The actual offset in the array will be used as the field tag; for protobuf compatibility, do not use offset 0.

_pack( format, array, bytebuf, pos )

Encode the data into the given buffer bytebuf starting at offset pos.p. bytebuf may also be an array.

unpack( format, bytebuf )

Decode the binary data in bytebuf according to the format. Returns an array of data items, each stored at the array offset that is the same as stored field tag.

_unpack( format, bytebuf, pos )

Dencode the data in bytes starting at position pos.p. bytebuf may also be an array.

Format

The pack/unpack format is a concatenated series of conversion specifiers, one per data item. Skipping items is not yet supported. The conversion specifiers were deliberately patterned after PERL pack().

For example, the format 'iVad' would store the first four data items from the input array, packing them as varint, uint32, string and double64, respectively. The number of data items converted is controlled by the format string.

As of 12/10/17:

'i': { wt: 0, enc: encodeVarint, dec: decodeVarint },       // sint
'I': { wt: 0, enc: encodeUVarint, dec: decodeUVarint },     // uint
'j'; { wt: 0, enc: encodeUVarint64, dec: decodeUVarint64 }, // int32, int64
'b': { wt: 0,                                               // bool
       enc: function(v, buf, pos) { buf[pos.p++] = v ? 1 : 0 },
       dec: function(buf, pos) { return buf[pos.p++] ? true : false } },
'd': { wt: 1, enc: encodeDouble, dec: decodeDouble },       // double
'q': { wt: 1, enc: encodeInt64, dec: decodeInt64 },         // sint64
'P': { wt: 1, enc: encodeUInt64, dec: decodeUInt64 },       // uint64
'a': { wt: 2, enc: encodeString, dec: decodeString },       // string
'Z': { wt: 2, enc: encodeBinary, dec: decodeBinary },       // binary
'f': { wt: 5, enc: encodeFloat, dec: decodeFloat },         // float
'l': { wt: 5, enc: encodeInt32, dec: decodeInt32 },         // sfixed32
'V': { wt: 5, enc: encodeUInt32, dec: decodeUInt32 },       // fixed32
// enum? (else int32)

The protocol-buffers prototype scalar value types are stored as

type          fmt   wt      
double        `d`   1       8 byte double
float         `f`   5       4 byte float
int32         `j`   0       signed varint.  Stores unsigned bits;
                            ie -1 stored as 8x 0xff.  On decode bits
                            are sign-extended.
int64         `j`   0       signed varint.  Stores unsigned bits;
                            on decode bits are sign-extended.
uint32        `I`   0       unsigned varint
uint64        `I`   0       unsigned varint
sint32        `i`   0       signed varint
sint64        `i`   0       signed varint
fixed32       `l`   5       always 4 bytes, ?decodes as unsigned?
fixed64       `q`   1       always 8 bytes
sfixed32      `l`   5       always 4 bytes, sign-extended on decode
sfixed64      `q`   1       always 8 bytes, sign-extended on decode
bool          `i`   0       varint 0 or 1
string        `a`   2       string, must be Utf-8 or 7-bit ASCII
bytes         `Z`   2       binary string

Todo

  • needs tests
  • verify that j and k encoding are match what is expected

Related Work

node-protobuf-lite's People

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.