bobbyrauchenberg / avronaut Goto Github PK

Schema is currently working, but i think the code could be cleaned up in the instances by moving to an AST

Magnolia would generate an AST which was then parsed to a schema, keeping the instances simple and clean and also making things like shuffling schema elements, handling annotations etc... a lot cleaner

Currently no date formats are supported

Need to decide which we'll support as default and add instances for

• schema

• encoder

• decoder

Schema tests are currently comparing Strings, this is fragile

On the whole we should be able to convert to json and compare json objects (this may not apply to, for example Union schemas, where we want to enforce field order)

When you give a default for a field in the schema creation, the value needs to be converted to a Java structure

Currently there is a case match on possible types to use for defaults in SchemaHelper.transformDefault - it would be nice to make it possible for users to add their own default conversions - possibly in the shape of a map from DefaultType -> JavaType (which in reality would probably be sth like Map[String, Any] => Any

But this could then be passed in, or added to some config object when schema generation happened

Currently all apis return Either[Error, T]. This can translate nicely to cats Validation instances, and given Magnolia iterates over all params we can build a very descriptive error report if a typeclass generation fails

A nice to have would be to be able to also have this in json format

We need schema / encoder / decoder instances for UUID

Currently Encoder and Decoder will run thru a load of extra steps each time they're called

Encoder generates an AvroSchema, and also translates to Avro in the magnolia step. We should

• pass in a schema
• translate direct to AvroF in typeclass generation and jst run a catamorphism in the parser (currently anamorphism)
• allow an encoder to be generated up front - like avro4s has ToRecord, then pass this to the parser when we have the record

will need to change this

def encode[A](a: A)(implicit encoder: Encoder[A], schema: AvroSchema[A]): Either[Error, GenericData.Record] =
    for {
      s       <- schema.schema
      encoded <- encoder.encode(a)
      genRec <- encoded match {
                 case AvroRecord(schema, values) =>
                   Parser(new GenericData.Record(s)).parse(AvroRoot(schema, values))
                 case _ => Error(s"Can only encode records, got $encoded").asLeft
               }
    } yield genRec

similar with Decoder which currently does everything on each invocation, need to split this so that the interface is sth like

trait Decoder[A] {

  def apply(operation: DecodeOperation): Result[A]

}

GenericRecord returns Any when you lookup values. I didn't want to pollute the API passing Any about. However given the way the typeclass derivation works, i wasn't able to write a Decoder instance for, for example List, without passing the Any value from the GenericRecord

As a solution I have written a parser which traverses the schema passed by the reader, and pulls the relevant values from the generic record, this keeps Any out of any public facing APIs

However because this step happens before Magnolia derives typeclass instances, I don't have access to default parameter values (as Magnolia provides these) at parse time. So the parser currently can fail to parse a field because there is no value in the provided GenericRecord, but the overall process needs to still succeed, setting the field that parsed tp type ParseFail, this means that when the Magnolia based Decoder runs, ParseFails are propogated and it is possible to provide a default value for those fields (if a default exists).

This feels wrong because:

1. ParseFail has been added as an AvroType to allow me to return Result[AvroType] from Parser
2. It feels dirty to fail but still pass the parsing step (tho this is a necessary evil atm)

It would be nice to find a cleaner way to do this.

Need schema / encoder / decoder instances for
NonEmptyList
NonEmptyMap

(maybe some others)

Currently blows up with stackoverflow

Worth looking at magnolia in more detail, as this should be handled by magnolia