Giter Club home page Giter Club logo

axon-avro-serializer's Introduction

axon-avro-serializer

incubating Codacy Badge Build Status sponsored Maven Central codecov

This extension to the Axon Framework aims to provide support for serialization of Axon events with the Apache Avro data format.

Avro is a schema-based data format that can be serialized to JSON or byte sequence, which is useful to minimize the disk space needed for your Axon event store and bandwidth used for transport. Messages can be validated against a schema version and Avro supports schema evolution by automatically determining compatibility modes between different revisions.

Please check our official documentation for more details.

Developers

License

This library is developed under

Apache 2.0 License

Sponsors and Customers

sponsored

axon-avro-serializer's People

Contributors

codacy-badger avatar dependabot-preview[bot] avatar dependabot[bot] avatar jangalinski avatar jonasenglerholisticon avatar zambrovski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

aholbreich

axon-avro-serializer's Issues

Serialize/deserialize special types in metadata.

Not as nice ... fails when I actually serialize/deserialize

I added a disabled test to the AvroMetaDataExtTest that fails when serializing LocalDate.
This is non trivial ... I would like to close this issue, so we at least have metadata support and then work on the metadata/bytes via a separate issue. thoughts?

Originally posted by @jangalinski in #108 (comment)

use axon-bom 4.5.2

currently, we use the holixon/axon-framework-bom, but meanwhile the official axon bom seems to be up to date and stable, so let's us it

include schemaId in MetaData

It is encoded in the SingleObject Byte array. But it can't hurt to include it as metadata ... easier to access and check.

provide testcontainers itest

to prove that serialization of events (and metadata) works correctly, set up a "real" itest based on the axon server docker image using testcontainers.

Axon Avro Serializer Support for Kafka

Hello,

In brief, All events are defined as Avro Versioned Schemas, and the Java Classes are Autogenerated (perfectly). Multiple axon-based spring boot applications consume these events. This works well with the axon event bus. Circumstance behoves me to employ Kafka as the event bus (auto-configured - axon-kafka-spring-boot-starter); however, events are neither dispatched nor consumed.

How do I go about this issue? Thanks.

provide confluent registry adapter osgi plugin

One of the main goals of this project is to provide a base framework for avro serialization with axonframework.
To use avro from the axon server dashboard, this requires an osgi plugin to be installed on the axon server.

While the concrete axon specific listeners plugin is part of holixon/axon-avro-serializer, the core functionality (configuring and accessing an confluent registry) should be part of this repo.

This way, we can later just exchange the core plugin (for example to use confluent registry) without providing multiple implementations for axon server.

So the task is: Parallel to the confluent rest client module, provide an osgi bundle that allows running a registry-adapter as a plugin.

Is blocked by: #59

Apicurio Registry Plugin and Adapter Unable to Resolve Avro Schema(s)

Hello again,

There's a newer issue concerning the Apicurio schema registry adapter and plugins provided as part of the project.

Screenshot 2022-11-15 at 16 38 45

The issue seems to originate from the Schema Registration Component. It uploads and versions my schemas as it should; however, it fails in applying the appropriate schema Id to the Avro Schema in my Apicurio Registry instance...

Screenshot 2022-11-15 at 16 44 46

So, while deserialising the axon message payload, the adapter looks up an artefact of a specific serialVersionUID, first from the cache, then the schema registry (if unavailable). The error now arises since the Schema Id in this instance is TestEvent while the adapter looks up a schema of id:: 6729430080489649277

This error also occurs in the axon-Avro-server-schema-registry-apicurio-plugin installed on the axon server instance as shown here...

Screenshot 2022-11-15 at 16 52 56

While Querying for events stored on the axon server, I get jumbled up records...

Screenshot 2022-11-15 at 16 55 14

I have tried looking for misconfiguration errors on my end, but everything I have implemented follows the documentation (https://www.holixon.io/axon-avro-serializer/snapshot/reference/index.html) to the letter.

Kindly get back to me as soon as possible on possible solutions.

Kind Regards,
Kahiga wa Kiguru.

Create intermediate representation based on generic record format

Acceptance Criteria

  • Have two modules: command and query
  • query module uses schema X
  • command module uses a newer schema with incompatible change (e.g. a field is added but no default is provided)
  • an upcaster exists that is installed in the query part and fixes the incompatibility by setting the default
  • the upcaster works on the level of generic record

rename root poms

The suffix "-root" is misleading, and I do not like the suffix "-parent". Since the intermediate poms are just used to structure the multi module layout and have no "meaning" nor will ever be imported by other projects, lets just append a "_" to make their names unique for the project.

Is this project abandoned?

Hi,
we were looking at using this project in combination with an Axon + Apache Kafka Event Bus setup, but it seems that the documentation is rather lacking and some basic functionality is not given.

For example, given the Getting started page, the @EnableAxonAvroSerializer annotation is not importable.

These are the dependencies that I've included (using Gradle instead of Maven):

implementation "io.holixon.axon.avro:axon-avro-serializer-spring:0.0.5"
implementation "io.holixon.avro:avro-registry-adapter-apicurio:0.0.5"

Has this project been abandoned?

provide osgi plugin for apicurio registry adapter

One of the main goals of this project is to provide a base framework for avro serialization with axonframework.
To use avro from the axon server dashboard, this requires an osgi plugin to be installed on the axon server.

While the concrete axon specific listeners plugin is part of holixon/axon-avro-serializer, the core functionality (configuring and accessing an apicurio registry) should be part of this repo.

This way, we can later just exchange the core plugin (for example to use confluent registry) without providing multiple implementations for axon server.

So the task is: Parallel to the apicurio rest client module, provide an osgi bundle that allows running a registry-adapter as a plugin.

serialize message payload as json

The avro single object bytes can not be displayed correctly in axon server cloud.

Idea: Serialize the payload as json and store the schemaId in metaData (see #73).

use registry adapter 0.1.0

we did a lot of changes, fixes and version updates on the registry adapter, so the next release of the serializer should use them.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.