Giter Club home page Giter Club logo

libserdes's Introduction

libserdes

Copyright(C) 2015-2016 Confluent Inc.

  • This software is licensed under the Apache 2.0 license.

libserdes is a schema-based serializer/deserializer C/C++ library with support for Avro and the Confluent Platform Schema Registry.

The library is aimed to be used in the streaming pipeline, e.g. Apache Kafka, to perform data serialization and deserialization with centrally managed schemas.

On the producer side the application only needs to provide the centrally managed schema name or id and libserdes will fetch and load the schema from the remote schema registry. The loaded schema will be used for producer side verification and serialization of Avro objects.

On the consumer side libserdes will automatically extract the schema id of consumed serialized data and fetch the required schemas from the remote schema registry to perform verification and deserialization back to an Avro object.

This functionality is designed to work in close integration with the Kafka consumer or producer, see the examples directory for integrations with the librdkafka client library.

The serialization defaults to Avro but is completely pluggable through the configuration object interface, allowing applications to use libserdes for any serialization type.

libserdes is licensed under Apache License 2.0

Requirements

On Debian/Ubuntu based systems: sudo apt-get install libjansson-dev libcurl-gnutls-dev

NOTE: The avro libraries needs to be installed manually or through Confluent's APT and YUM repositories

Build

./configure
make
sudo make install

Documentation

Full API documentation is available in serdes.h and serdescpp.h

Configuration

libserdes typically only needs to be configured with a list of URLs to the schema registries, all other configuration is optional.

  • schema.registry.url - comma separated list of schema registry base URLs (no default)
  • deserializer.framing - expected framing format when deserializing data: none or cp1 (Confluent Platform framing). (default: cp1)
  • serializer.framing - framing format inserted when serializing data: none or cp1 (Confluent Platform framing). (default: cp1)
  • debug - enable/disable debugging with all or none. (default: none)

libserdes's People

Contributors

maxzheng avatar edenhill avatar xiangxin72 avatar ah- avatar beeender avatar drdo avatar worms avatar

Stargazers

 avatar

Watchers

James Cloos avatar  avatar  avatar

Forkers

qpc-github

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.