Giter Club home page Giter Club logo

pods's Introduction

Build Status Build status Coverage Status

Plain Old Data Serializer (PODS)

Feature List

  • header only
  • high performance (perhaps binary serialization is the fastest on the Earth)
  • optional values
  • supported archive formats:
    • JSON
    • MsgPack
  • serialization from/to:
    • memory buffer
    • resizable memory buffer
    • standard C++ streams

Benchmarks

Comparison with memcpy

MacBook Pro, 2017 3.5 GHz Intel Core i7 16 Gb 2133 MHz LPDDR3

data size: 11520
serialized data size: 11632

memcpy
    total: 27465 Mb
    total time: 2396.19 ms
    speed: 11461.9 Mb/s

serialization
    total: 27732 Mb
    total time: 3502.93 ms
    speed: 7916.8 Mb/s

deserialization
    total: 27732 Mb
    total time: 3200.15 ms
    speed: 8665.83 Mb/s

Comparison with other libraries

performing 100000 iterations

thrift-binary: version = 0.10.0
thrift-binary: size = 17017 bytes
thrift-binary: time = 2259 milliseconds

protobuf: version = 3001000
protobuf: size = 16116 bytes
protobuf: time = 2797 milliseconds

capnproto: version = 6001
capnproto: size = 17768 bytes
capnproto: time = 486 milliseconds

boost: version = 106200
boost: size = 17470 bytes
boost: time = 1365 milliseconds

msgpack: version = 2.1.3
msgpack: size = 13402 bytes
msgpack: time = 3638 milliseconds

cereal: size = 17416 bytes
cereal: time = 1034 milliseconds

avro: size = 16384 bytes
avro: time = 4780 milliseconds

flatbuffers: size = 17632 bytes
flatbuffers: time = 433 milliseconds

yas: version = 6.0.2
yas: size = 17416 bytes
yas: time = 317 milliseconds

pods: size = 17012 bytes
pods: time = 235 milliseconds

Using PODS

MsgPack serialization

#include <iostream>

#include <pods/pods.h>
#include <pods/msgpack.h>
#include <pods/buffers.h>

// just a struct for serialization
struct Server
{
    std::string address;        // no default value
    uint16_t port = 8080;       // default value

    PODS_SERIALIZABLE(
        PODS_MDR(address),      // mandatory field
        PODS_OPT(port))         // optional field
};

int main(int /*argc*/, char** /*argv*/)
{
    const Server original;

    pods::ResizableOutputBuffer out;
    pods::MsgPackSerializer<decltype(out)> serializer(out);
    if (serializer.save(original) != pods::Error::NoError)
    {
        std::cerr << "serialization error\n";
        return EXIT_FAILURE;
    }

    Server loaded = {};

    pods::InputBuffer in(out.data(), out.size());
    pods::MsgPackDeserializer<decltype(in)> deserializer(in);
    if (deserializer.load(loaded) != pods::Error::NoError)
    {
        std::cerr << "deserialization error\n";
        return EXIT_FAILURE;
    }

    if (original.address != loaded.address
        || original.port != loaded.port)
    {
        std::cerr << "corrupted archive\n";
        return EXIT_FAILURE;
    }

    return EXIT_SUCCESS;
}

JSON serialization

#include <iostream>

#include <pods/pods.h>
#include <pods/json.h>
#include <pods/buffers.h>

// just a struct for serialization
struct Server
{
    std::string address;        // no default value
    uint16_t port = 8080;       // default value

    PODS_SERIALIZABLE(
        PODS_MDR(address),      // mandatory field
        PODS_OPT(port))         // optional field
};

struct ServerList
{
    std::vector<Server> servers =
    {
        // this is default values
        Server { "localhost", 8080 },
        Server { "my.com", 2018 }
    };

    PODS_SERIALIZABLE(
        PODS_MDR(servers))
};

int main(int /*argc*/, char** /*argv*/)
{
    const ServerList original;

    pods::ResizableOutputBuffer out;
    pods::PrettyJsonSerializer<decltype(out)> serializer(out);
    if (serializer.save(original) != pods::Error::NoError)
    {
        std::cerr << "serialization error\n";
        return EXIT_FAILURE;
    }

    ServerList loaded;
    loaded.servers.clear();

    pods::InputBuffer in(out.data(), out.size());
    pods::JsonDeserializer<decltype(in)> deserializer(in);
    if (deserializer.load(loaded) != pods::Error::NoError)
    {
        std::cerr << "deserialization error\n";
        return EXIT_FAILURE;
    }

    const std::string json(out.data(), out.size());
    std::cout << json << '\n';

    return EXIT_SUCCESS;
}
Output:
{
    "servers": [
        {
            "address": "localhost",
            "port": 8080
        },
        {
            "address": "my.com",
            "port": 2018
        }
    ]
}

Standard streams

#include <iostream>
#include <sstream>

#include <pods/pods.h>
#include <pods/msgpack.h>
#include <pods/streams.h>

// just a struct for serialization
struct Server
{
    std::string address;        // no default value
    uint16_t port = 8080;       // default value

    PODS_SERIALIZABLE(
        PODS_MDR(address),      // mandatory field
        PODS_OPT(port))         // optional field
};

int main(int /*argc*/, char** /*argv*/)
{
    const Server original;

    std::stringstream buffer;

    pods::OutputStream out(buffer);
    pods::MsgPackSerializer<decltype(out)> serializer(out);
    if (serializer.save(original) != pods::Error::NoError)
    {
        std::cerr << "serialization error\n";
        return EXIT_FAILURE;
    }

    Server loaded = {};

    pods::InputStream in(buffer);
    pods::MsgPackDeserializer<decltype(in)> deserializer(in);
    if (deserializer.load(loaded) != pods::Error::NoError)
    {
        std::cerr << "deserialization error\n";
        return EXIT_FAILURE;
    }

    return EXIT_SUCCESS;
}

pods's People

Contributors

chiezero avatar mtrempoltsev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

pods's Issues

Add Json DOM parser

This is need for field order independent parsing. More slower but more friendly for manual Json editing.

Ускорить ReadOnlyStreamStorage

WriteOnlyStreamStorage использует буфер и его производительность хороша, но ReadOnlyStreamStorage не имеет буфера и читает напрямую из std::istream, поэтому производительность неудовлетворительна. Нужно использовать буфер.

Добавить тип Blob

Чтобы в JSON сериализовать бинарные данные нужно добавить специальный тип. Пример:

struct Blob
{
    char* data;
    size_t size;
};

Добавить еще один тип Binary

Сейчас можно сериализовать бинарные данные указав тип-обертку Binary. Она по сути указатель на фиксированный блок памяти, но иногда надо чтобы при десереализации можно было увеличить размер памяти.

Consider changing (or redocumenting) the example on the main page

Hi,

I haven't used PODs before, so the first place I'm looking at documentation to get an idea of how difficult or eays it will be ot integrate it into my application is the example on the main readme in the repo.

When I first glossed over it, considering a few different libraries ot use for my project, I got the impression that PODS required a server internally to function - perhaps underneath, the library was setting one up to receive data in the background, etc. etc.

I now realise that this is just because the sort of data we are storing is a list of servers, but this wasn't immediately obvious. Therefore, I think it would be a good idea to add a note on this line explaining what we're doing, or pick a different example of domain data to show on the main page of the repository (something more obvious as an example., like fruits, animals or planets), as without having seen the library before, intermingling Server code and PODS code, it gave me the impression PODs required some sort of more complex Server structure in order to work, and thus it would need a lot more work involved in getting it to work than it really will.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.