Giter Club home page Giter Club logo

atd's Introduction

ATD project - Static Types for Json APIs

CircleCI

ATD stands for Adaptable Type Definitions. It is a syntax for defining cross-language data types. It is used as input to generate efficient and type-safe serializers, deserializers and validators.

Target programming languages currently supported:

Export to other interface description languages (IDLs):

Tools to work with ATD files:

  • atdcat: check syntax, reformat, expand inherited definitions, export to JSON Schema, ...
  • atddiff: compare two revisions of a given ATD interface and report incompatibilities.

All can installed with opam e.g.

$ opam install atdgen

Documentation

here

Contributing

The ATD suite of tools is developed and maintained by volunteers—users like you. Various issues are in need of attention. If you'd like to contribute, please leave a comment on the issue you're interested in, or create a new issue. Experienced contributors will guide you as needed.

There are many simple ways of making a positive impact. For example, you can...

  • Use the software in your project.
  • Give a demo to your colleagues.
  • Share the passion on your blog.
  • Tweet about what you're doing with atd.
  • Report difficulties by creating new issues. We'll triage them.
  • Ask questions on StackOverflow.
  • Answer questions on StackOverflow.
  • Discuss usage on the OCaml forums.
  • Pick a task that's easy for you.

Check out in particular good first time issues and other issues with which we could use some help.

For guidelines on how to contribute code, consult CONTRIBUTING.md.

Authors

The ATD project started in 2010. Contributors include:

  • Martin Jambon
  • Rudi Grinberg
  • Alexandre Bourquelot
  • Ivan Jager
  • oleksiy
  • Gregoire Lionnet
  • David Sheets
  • Rytis Jonynas
  • Jeff Meister
  • Carmelo Piccione
  • Louis
  • Louis Roché
  • Raman Varabets
  • Daniel Weil
  • Egor Chemokhonenko
  • Gabriel Scherer
  • Louis Roché (Ahrefs)
  • Matthew McQuaid
  • koonwen
  • tzm
  • Mathieu Baudet
  • Oleksiy Golovko
  • Rauan Mayemir
  • Hyeseong Kim
  • John Billings
  • Marek Kubica
  • Zach Yannes
  • Antonin Décimo
  • Brendan Long
  • Caio Wakamatsu
  • Chris Yocum
  • Pierre Boutillier
  • Shon Feder
  • metanivek
  • sebastiantoh
  • Anurag Soni
  • Arjun Ravi Narayan
  • Asya-kawai
  • Christophe Troestler
  • Damien Doligez
  • Daniel M
  • Enrico Tassi
  • François Pottier
  • Javier Chavarri
  • Jonas Bergler
  • Kate
  • Stephane Legrand
  • Vincent Bernardoff
  • Zach
  • haoyang
  • pmundkur
  • ygrek

We distribute the source code under the terms of a BSD license.

atd's People

Contributors

aij avatar brendanlong avatar caiowakamatsuahrefs avatar cometkim avatar cyberhuman avatar cyocum avatar dsheets avatar dweil avatar elrandar avatar gasche avatar ixzzd avatar jameister avatar jchavarri avatar jnb avatar khady avatar koonwen avatar leonidas-from-xiv avatar ma2bd avatar mbarbin avatar metanivek avatar misterda avatar mjambon avatar mmcqd avatar pirbo avatar rauanmayemir avatar rgrinberg avatar sebastiantoh avatar struktured avatar syntakker avatar zindel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

atd's Issues

Get rid of Obj.magic for record readers

From @mjambon on June 18, 2016 9:51

This is a followup to https://github.com/mjambon/atdgen/issues/45 and mjambon/atdgen#46.

Get rid of Obj.magic for record readers and instead, use plain old option refs:

let x = ref None in
let y = ref None in
...
| "x" -> x := Some (read_x_type ...)
| "y" -> y := Some (read_y_type ...)
...
{
  (* optional field *)
  x = !x;
  (* required field *)
  y = (match !y with
      | None -> ... (* error *)
      | Some v -> v
}

Perhaps fields with a precomputed default could use a ref instead of an option ref:

let y = ref the_default in
...
| "y" -> y := read_y_type ...
...

but only if the_default is a precomputed value, since we don't want to compute a default value for nothing.

Copied from original issue: mjambon/atdgen#51

Parsing to different value constructors of a variant type based on a field's value

From @sgronblo on October 14, 2016 1:17

Maybe I'm too stupid to figure this out, but I would like to be able to parse the following two json values:

{
  "type": "twod-point",
  "x": 4,
  "y": 10
}

{ "type": "threed-point",
  "x": 5,
  "y": 3,
  "z": 8
}

Into the following types and values:

type Point = TwoDPoint of (int * int) | ThreeDPoint (int * int * int)

let twodpoint = TwoDPoint 4 10
let threedpoint = ThreeDPoint 5 3 8

Bonus points if you would be able to map the string value contained in the type field to any value constructor name you want. But even some mapping by naming convention would be ok.

Copied from original issue: mjambon/atdgen#59

Do not generate unused functions

From @rgrinberg on June 12, 2016 0:34

Seems like atdgen generates unused definitions like:

let write__1 = (
  Ag_oj_run.write_list (
    Ag_oj_run.write_float_as_int
  )
)
let string_of__1 ?(len = 1024) x =
  let ob = Bi_outbuf.create len in
  write__1 ob x;
  Bi_outbuf.contents ob
let read__1 = (
  Ag_oj_run.read_list (
    Ag_oj_run.read_number
  )
)
let _1_of_string s =
  read__1 (Yojson.Safe.init_lexer ()) (Lexing.from_string s)
let write_unixtime_list = (
  write__1
)

Is there a way to get rid of these perhaps?

Copied from original issue: mjambon/atdgen#49

Problems building manual

  1. Missing link to atd_version.ml

    09:04:48 alex@p50:~/git/atd/manual (master 214474b*)$ make
    make: *** No rule to make target '../atd_version.ml', needed by 'atd-manual.tex'. Stop.

    I fixed it as follows in my copy of the repo:

cp -s ./_build/install/default/lib/atd/atd_version.ml .

  1. camlmix complains about an unbound value:

    09:06:42 alex@p50:~/git/atd/manual (master 214474b*)$ make
    OCAMLPATH=../..:$OCAMLPATH camlmix atd-manual.mlx -o atd-manual.tex
    OCAMLPATH=../..:$OCAMLPATH camlmix atd-body.mlx -o atd-body.tex
    File "atd-body.mlx", line 18, characters 3-6:
    Error: Unbound value atd
    Makefile:18: recipe for target 'atd-body.tex' failed
    make: *** [atd-body.tex] Error 2

Put remaining fields/values in a "grab-bag" Yojson.Basic.json type field?

From @sgrove on August 27, 2017 7:42

It'd be nice if I could list keys I know explicitly ahead of time:

type myObj = {
  id: string;
  ?description: string option;

but also have a "and any other fields I haven't listed should be put under this field":

  dynamic <json capture_all="true">: json
}

Is there any way to approximate this? There's an API I'm working with that has lots of static keys, and then a lot of dynamic keys. I'd like to use ATD for all its conveniences, but be able to drop down to yojson when I need to.

Copied from original issue: mjambon/atdgen#70

git tag naming

Hi,

the most recent atd version was named with a different scheme compared to the last releases. It misses a "v" in the front. Was this just a mistake or will tags now not have a "v" in the front anymore?

Context: I'm the Debian maintainer for atd and am maintaining the regex that checks the github page for new releases.

Thanks!

cheers, josch

No rule found for atd.version.sexp

The installation of the atdgen package fails when it's pinned to the local development folder (master branch).

$ opam pin add atdgen .
...
No rule found for atd.version.sexp

which comes from:

$ jbuilder build -p atdgen
No rule found for atd.version.sexp

The atd package uses an equivalent command (jbuilder build -p atd) but it succeeds. @rgrinberg, would you happen to know where atd.version.sexp comes from?

`open' construct for ATD?

From @alexbaretta on September 8, 2017 16:4

I have a project whose main data types are defined in several ATD files. Currently I'm using the following syntax to allow one ATD file to refer to type defined in another:

type foo_t <ocaml_json module="Foo"> = abstract

This is highly specific to Ocaml however. Indeed, the type structure is only fully defined in OCaml, as the types are declared abstract in ATD. So is there a way in pure ATD, regardless of the target language, to express a dependency between ATD modules, something akin to the 'open' construct in OCaml or to the '#include' construct in C?

Copied from original issue: mjambon/atdgen#71

Polymorphic type inside wrap doesn't get type dependencies right

From @akotulski on April 19, 2017 16:52

When I tried to make type t inside wrap use polymorphic type it amazingly works! However, generated code doesn't know about dependencies between types. Adding -rec option to atdgen helps, but I was wondering if there is better way to specify such relationship.

Very simplified example (I'm not using list nor int in real life situation):

(* polymorphic.atd *)
type t1 = int
type t2 = int wrap <ocaml t="t1 list" wrap="fun x -> [x]" unwrap="List.hd">
(* polymorphic_t.ml *)
(* Auto-generated from "polymorphic.atd" *)


type t2 = t1 list

type t1 = int

This leads to compilation errors:

File "polymorphic_t.ml", line 4, characters 10-12:
Error: Unbound type constructor t1

Copied from original issue: mjambon/atdgen#63

Full support for variants

From @gerdstolpmann on December 9, 2015 15:18

Currently, atdgen only supports unary variants. E.g. atd

type x = [ X of (int * string) ]

is mapped to OCaml

type x = [ `X of (int * string) ]

and not to

type x = [ `X of int * string ]

This is a show-stopper when you try to serialize existing types, and you cannot change the type definition. This is in particular an issue because unary variants of tuples are rather uncommon in the OCaml world.

Copied from original issue: mjambon/atdgen#40

Cleanly re-organize atdgen tests

atdgen/test is a total mess at the moment (and has always been). We should have a way to (1) know where a specific feature is tested, (2) know where to add tests for a new feature, and (3) produce a clear report of which tests succeeded and which ones failed.

Document the structure of the repo

Each file of the atd codebase should start with a comment that explains why it exists, what it does, and how it fits with the rest of the codebase.

It's important for potential contributors to easily understand how the project is organized so that they can fix or improve just the part they want without having to reverse engineer the whole project.

Warning 32: unused value string_of__1

Those warnings can even be found in RWO v1. If I run jbuilder with --dev making it use stricter compilation flags by default, I get multiple warnings like:

File "src/types_j.ml", line 25, characters 4-16:
Warning 32: unused value string_of__1.
File "src/types_j.ml", line 597, characters 4-16:
Warning 32: unused value string_of__2.
File "src/types_j.ml", line 613, characters 4-16:
Warning 32: unused value string_of__3.
File "src/types_j.ml", line 1:
Error: Some fatal warnings were triggered (3 occurrences)

I added an example to reproduce the issue here: https://github.com/rauanmayemir/atdgen-example

CC: @rgrinberg

Allow ocaml wrap types to be optional

From @trevorsummerssmith on June 6, 2015 19:51

Currently it is not possible to provide a wrap type that creates an option that is an atd option type.

eg The following does not work:

type a = { ?field : string option wrap <ocaml t="My.t option" wrap="My.wrap" unwrap="My.unwrap">; }

It would be very useful to allow this behavior as many third party APIs need wrapping to work. I haven't looked into how difficult this would be to implement. Is there any interest in this feature?

Thanks.

Copied from original issue: mjambon/atdgen#35

Better validator error messages?

I was trying to use atdgen to validate fields in a record, similar to the example in the docs:

type t =
  { id : string
  ; start_date : date
  ; ?end_date : date option }
  <ocaml valid="Requisition_util.validate_requisition">
(* requisition_util.ml *)
let validate_requisition { Requisition_t.start_date ; end_date } =
  match end_date with
  | None -> true
  | Some end_date -> Date.(end_date >= start_date)

The problem is that the error message is just:

Validation error; path = <root>

I looked at the code in atdgen/src/validate.ml, and it doesn't seem like there's any way to set msg. Is there some way to make the message more useful? I'd really like to return something like "end_date must be >= to start_date".

Support for interpreting {"x":null} as {x = Some None}

The goal of this feature is to support patch semantics for records. A patch record is a set of optional fields. The presence of a field in json indicates how to modify a certain field in some destination object. Given that the destination object itself may already have optional fields, we need a way to express "clear this field's value".

The problem is that until now, atdgen was treating a null json field as if the field was missing. Instead here, we want the null value of the field to indicate a request to clear or delete a field.

(* Type of the objects stored in our database *)
type t = {
  ?x : int option;
  ?y : int option;
  ?z : int option;
}
(* Request to modify some of the fields of an object *)
type t_patch = {
  ?x : int nullable option;
  ?y : int nullable option;
  ?z : int nullable option;
} <ocaml field_prefix="patch_"> <json keep_nulls>

Let's consider the following json patch:

{
  "x": 1, // request to set field x to 1
  "y": null // request to clear field y
  // implicit request to leave field z unchanged
}

It has the following OCaml representation:

{
  patch_x = Some (Some 1);
  patch_y = Some None;
  patch_z = None;
}

This gets us two levels of optionality (_ option option) in json, while staying idiomatic.

I'm about to implement this keep_nulls feature.

Idea: Use cmdliner for new command API

From @rgrinberg on June 12, 2016 3:23

I noticed that you're looking to revamp the CLI of atden. May I recommend to use cmdliner. It's quite a nice library for creating consistent CLI's (opam is one example) and has many nice features such as automatic man page generation. One good example of cmdliner's features is opam itself.

If you're interested I can whip up a quick demo of how to do a subcommand like interface that you're looking for in atdgen.

Copied from original issue: mjambon/atdgen#50

Plan for more flexible support for variants + getting rid of Obj.magic

Many json APIs out there use various conventions for representing sum types (variants), and supporting them all has been a pain point for all consumers of external APIs. Support for one specific convention is the tag_field annotation that was added to atdgen by @dsheets. Unfortunately, many other conventions also need to be supported. This question was raised again recently by users and I encountered it personally. I now have a good solution to that problem, which involves letting the user transform the json tree to make it atd-compliant. Let's call such code "json adapters".

The planned upcoming features for atdgen are:

  1. Implement support for json adapters.
  2. Emulate the tag_field feature using a json adapter and get rid of its legacy implementation.
  3. Eliminate the use of the Obj module from all code generated by atdgen for json handling.

(1) was started about a week ago. (3) was started a while ago but stopped due to a lack of resources and because of extra work due to code duplication introduced by tag_field support. Getting rid of Obj.magic will reduce maintenance costs and will give users a greater peace of mind, but it's dependent on the simplifications brought by (2).

Please keep in mind that this work is done on my free time, and I don't have much of it. More specifically, I give myself 3 more weekends to get this done (end of April 2018) before returning to other side projects.

Unbound value Yojson.Safe.validate_json

From @vbmithr on December 16, 2015 9:9

When using, in an .atd file,

type json <ocaml module="Yojson.Safe"> = abstract

and using atdgen -t file.atd, a call to Yojson.Safe.validate_json gets generated. This probably shouldn’t be.

Copied from original issue: mjambon/atdgen#41

Double `<ocaml ...> <ocaml ...>` annotation not merged properly

From @mjambon on December 15, 2016 0:7

Reported by @alexbaretta.

I'm using <ocaml predef> to reference a type defined in a separate atd file:

type point <ocaml from="Proto"> <ocaml predef> = {
  s_gamma : float ;
  s_n : float ;
  s_loss  : float ;
}

This compiles into two different type definitions in the _t and _j files.
_j.mli:

type point = Proto_t.point = { s_gamma: float; s_n: float; s_loss: float }

Which seems right to me.
_t.mli:

type point = { s_gamma: float; s_n: float; s_loss: float }

This version lacks does not equate type point defined locally to type point defined in Proto_t, causing the Ocaml compiler to barf:

Any idea?

Copied from original issue: mjambon/atdgen#60

Remove -o-name-overlap and -o-no-name-overlap options

As far as I understand, these options are only useful for obsolete versions of OCaml: < 4.01. It's doubtful that atd even works on those versions since it doesn't even build. Dune only works on >= 4.02 anyway.

Split runtime into own opam package

There's a use case where users would like to vendor the generated code by atdgen into their repos. Unfortunately, this doesn't let you drop the dependency on atdgen as the runtime is still in atdgen itself. It would be nice if this was a separate package to allow this kind of use case. I'm proposing we split out the runtime into a atdgen-runtime package for this purpose.

`open' construct for ATD?

I'm copying this issue from atdgen as I have been told atdgen has now been merged with atd.

My question:
I have a project whose main data types are defined in several ATD files. Currently I'm using the following syntax to allow one ATD file to refer to type defined in another:

type foo_t <ocaml_json module="Foo"> = abstract

This is highly specific to Ocaml however. Indeed, the type structure is only fully defined in OCaml, as the types are declared abstract in ATD. So is there a way in pure ATD, regardless of the target language, to express a dependency between ATD modules, something akin to the 'open' construct in OCaml or to the '#include' construct in C?

Martin's answer
No, there isn't. I don't have a good sense how what atd modules would canonically turn into for languages other than OCaml.

Note that <ocaml_json module=...> is obsolete. You should use just instead.

Note also that we moved the atdgen repository into atd. It's best to discuss things there instead.

Don't generate unused module items such as `string_of__27`

This would reduce generated code size and would allow the user to compile their code with warning 32 enabled and without having to make an exception for atdgen-generated code. Right now we're getting warnings like the following:

File "foo_j.ml", line 6015, characters 4-17:
Warning 32: unused value string_of__27.

We're also getting warnings for validate__ functions:

File "foo_v.ml", line 525, characters 4-16:
Warning 32: unused value validate__18.

Multi-job builds fail with 'Unbound module' errors

From @ntnn on March 22, 2017 22:1

Asynchronous builds fail with missing dependencies in the makefile src/Makefile.

Expected behaviour: Successful compiation
Actual behaviour: Failing build with Unbound module errors

Steps to reproduce:

  1. Clone current master
  2. make --jobs 4
  3. Failing build

Example output:

$ make -j 4
make -C src
make[1]: Entering directory '/tmp/ntnn/atdgen/src'
echo 'let version = "1.10.0"' > ag_version.ml
ocamlfind ocamldep -package "str atd biniou yojson" ag_doc.mli ag_string_match.mli ag_util.mli ag_version.ml ag_error.ml ag_mapping.ml ag_doc_lexer.ml ag_doc.ml ag_ocaml.ml ag_indent.ml ag_ox_emit.ml ag_biniou.ml ag_xb_emit.ml ag_ob_mapping.ml ag_ob_spe.ml ag_ob_emit.ml ag_string_match.ml ag_json.ml ag_oj_mapping.ml ag_oj_emit.ml ag_validate.ml ag_ov_mapping.ml ag_ov_emit.ml ag_ob_run.ml ag_oj_run.ml ag_ov_run.ml ag_util.ml > dep
echo 1.10.0 > VERSION
echo 'version = "1.10.0"' > META
ocamllex ag_doc_lexer.mll
cat META.in >> META
46 states, 313 transitions, table size 1528 bytes
make dep
make[2]: Entering directory '/tmp/ntnn/atdgen/src'
make[2]: 'dep' is up to date.
make[2]: Leaving directory '/tmp/ntnn/atdgen/src'
make atdgen.cma atdgen.run
make atdgen.cmxa atdgen.cmxs atdgen
make[2]: Entering directory '/tmp/ntnn/atdgen/src'
make[2]: Entering directory '/tmp/ntnn/atdgen/src'
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_version.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_error.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_mapping.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_version.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_doc_lexer.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_error.ml
File "ag_mapping.ml", line 3, characters 5-13:
Error: Unbound module Ag_error
make[2]: *** [Makefile:150: ag_mapping.cmi] Error 2
make[2]: *** Waiting for unfinished jobs....
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_mapping.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_doc_lexer.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_doc.mli
make[2]: Leaving directory '/tmp/ntnn/atdgen/src'
make[1]: *** [Makefile:103: opt] Error 2
make[1]: *** Waiting for unfinished jobs....
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_ocaml.mli
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_indent.ml
ocamlfind ocamlc -dtypes -g -c -package "str atd biniou yojson" ag_ox_emit.mli
File "ag_ocaml.mli", line 101, characters 11-38:
Error: Unbound module Ag_mapping
make[2]: *** [Makefile:147: ag_ocaml.cmi] Error 2
make[2]: *** Waiting for unfinished jobs....
File "ag_ox_emit.mli", line 1, characters 16-39:
Error: Unbound module Ag_ocaml
make[2]: *** [Makefile:147: ag_ox_emit.cmi] Error 2
make[2]: Leaving directory '/tmp/ntnn/atdgen/src'
make[1]: *** [Makefile:101: all] Error 2
make[1]: Leaving directory '/tmp/ntnn/atdgen/src'
make: *** [Makefile:13: default] Error 2

The following patch allows multi-job builds:

diff --git a/src/Makefile b/src/Makefile
index b43859b..d0e2688 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -143,10 +143,20 @@ META: META.in Makefile
 VERSION: Makefile
 	echo $(VERSION) > VERSION
 
-%.cmi: %.mli
+ag_mapping.cmi: | ag_error.cmi
+ag_ob_emit.cmi: | ag_ob_spe.cmi ag_ov_emit.cmi ag_ov_mapping.cmi ag_mapping.cmi
+ag_ob_spe.cmi: | ag_ob_mapping.cmi
+ag_ov_emit.cmi: | ag_ov_mapping.cmi
+ag_oj_mapping.cmi: | ag_json.cmi ag_ocaml.cmi
+ag_ob_mapping.cmi: | ag_ocaml.cmi
+ag_ov_mapping.cmi: | ag_ocaml.cmi
+ag_ocaml.cmi: | ag_doc.cmi
+ag_ox_emit.cmi: | ag_ocaml.cmi
+
+%.cmi: %.mli | ag_mapping.cmi
 	ocamlfind ocamlc $(OCAMLFLAGS) -c -package "$(OCAMLPACKS)" $<
 
-%.cmi: %.ml
+%.cmi: %.ml | ag_mapping.cmi
 	ocamlfind ocamlc $(OCAMLFLAGS) -c -package "$(OCAMLPACKS)" $<
 
 %.cmo: %.ml

I'm sure the dependencies can be resolved in a nicer fashion, but I don't know OCaml.

Copied from original issue: mjambon/atdgen#62

Also provide /usr/bin/atdgen without ocamlopt

From @josch on September 15, 2016 15:43

Hi,

it seems that currently, an executable called atdgen is only installed into $(BINDIR)/ if the opt target gets executed beforehand. This means that on architectures without ocamlopt, $(BINDIR)/ will contain a different interface than on architectures with ocamlopt. This in turn means, that an architecture-independent other program cannot rely on the existance of, for example, /usr/bin/atdgen but also has to test for /usr/bin/atdgen.run.

Would it be possible for atdgen to provide an architecture independent interface to the atdgen executable? That way, users of atdgen would not have to check which executable is available but could trust a single interface.

For example, it would be possible to install a symlink from atdgen to atdgen.run on platforms that come without ocamlopt.

Would that be possible?

Copied from original issue: mjambon/atdgen#58

Make all call to ocaml through ocamfind

Repository whitequark/opam-cross-windows offers a way to cross compile ocaml code for windows. It works by providing an alternative toolchain to ocamlfind so that "OCAMLFIND_TOOLCHAIN=windows" ocamlfind opt ... generates windows binaries.

For this purpose, it is convenient that any call to ocaml* goes through ocamlfind which is the case here everywhere but for 2 instances. I've made the trivial patch:
https://github.com/whitequark/opam-cross-windows/blob/master/packages/atd-windows.1.2.0/files/patches/ocamlc_config_through_ocamlfind.patch and I forward it here for inclusion (if relevant).

atdgen can put types in the wrong order

I'm running into an issue where my atd definition will succeed in generating an ml file of types (atdgen -t), but the file won't compile.

I managed to reduce it to the following. If I remove any of the definitions, then compiling the generated test_t.ml file succeeds.

type sku = {
  product: product;
}

type product = {
  sku: sku;
}

type plan = {
  product: product;
}

type customer = {
  plan: plan;
  customer: customer;
}

type charge = {
  order: order;
}

type order = {
  charge: charge;
  sku: sku;
}

The generated file (atdgen -t test.atd):

(* Auto-generated from "test.atd" *)

type plan = { product: product }

type customer = { plan: plan; customer: customer }

type product = { sku: sku }

and sku = { product: product }

type charge = { order: order }

and order = { charge: charge; sku: sku }
$ ocamlc src/test_t.ml
File "src/test_t.ml", line 4, characters 23-30:
Error: Unbound type constructor product

A way to get the unhandled fields in a JSON object?

I'm using atdgen to parse iCIMS types, and I'd like to make our parsing less specific to our iCIMS integration.

https://developer.icims.com/REST-API/Object-Types-Commands/Profiles

Each type has a set of fields that always exist (id, firstname, lastname, etc.), which I'd like to parse with atdgen, but there's also a set of custom fields, with names like field1234. These custom fields are user-specific, and there are thousands of them, so I just want them to be returned as string -> string map.

I can get atdgen to return the whole object as raw JSON, but is there a way for it to parse what it can and then return the rest?

Ideally I'd be able to do something like:

let { Icimst.firstname ; lastname ; other } = Icims_j.person_of_string some_string in
let custom =
  List.filter other ~f:(fun (key, _) -> String.has_prefix key ~prefix:"field")
  |> String.Map.of_alist_exn
in
{ first_name = firstname ; last_name = lastname ; custom }

Any interest in a jbuilder of atdgen and its dependencies?

From @rgrinberg on May 13, 2017 23:35

https://github.com/janestreet/jbuilder is a new build system from janestreet. It's fast, easy to use, well documented, portable, and has no dependencies. The OCaml community is converging fast on it.

But my own reason for porting atdgen and its dependencies to it, is that it makes vendoring code very easy. If you'd like, I can do the porting work for atdgen and friends.

Copied from original issue: mjambon/atdgen#64

OPAM installation

Hello,
I'm not sure this is the right place to report this?
I tried to install opam install atdj but it failed.

It seems that the old repository is pointed to by the some opam file?

Eventually, the simplest way I found has been:

opam pin add atd.1.13.0 https://github.com/mjambon/atd.git
opam pin add atdgen-runtime.1.13.0 https://github.com/mjambon/atd.git
opam pin add atdgen.1.13.0 https://github.com/mjambon/atd.git
opam pin add atdj.1.13.0 https://github.com/mjambon/atd.git

Salutations

atdgen 1.10.2 causes infer 0.7.0 build breakage

From @jayvdb on May 25, 2017 18:48

https://github.com/coala/coala-bears builds have been using infer 0.7.0 which built happily on Circle CI and Travis CI using atdgen 1.10.0 with ocaml 4.02.3/opam 1.2.2 from Ubuntu Trusty.

This broken recently, and we just noticed as our builds use a cached build of infer. Our issue for the breakage is coala/coala-bears#1763 .

The most obvious difference between old and new builds is that atdgen is now 1.10.2.

Building the latest version of infer from source works, but that is likely just working around the problem as it installed atdgen 1.6.0.

The fix (umerged coala/coala-bears#1765) pins atdgen to 1.10.0, and all is well when (I havent analysed this part very much) facebook's reason is also upgraded.

coala/coala-bears@6104b3f63d7efe11961 is the current master with the problem, and can be used to reproduce the problem even if we merge a workaround, but those builds are very long as it has a lot of crazy dependencies.

Copied from original issue: mjambon/atdgen#68

New release for the cmxs

Is it possible to have a new release of atd, that would include the build of the cmxs ? Currently I have to pin the git repo on OPAM.

Thanks

Generate annotations to ignore warnings

From @rgrinberg on June 11, 2016 22:59

Atdgen generates code (AFAIK it's only the json converted code) that triggers some OCaml warnings. For example the unused value warning.

it would be great if atdgen would generate teh appropriate [@@ocaml.warning ...] annotations to silence these warnings. Since atdgen supports older OCaml then this should be optional but perhaps it's worth it to make it the default as pre ppx OCaml is dying quickly.

Copied from original issue: mjambon/atdgen#47

Putting field doc text into source code so it can be accessed at run time?

From @sgrove on March 12, 2018 6:36

I've filled out a lot of atd definitions with <doc text="..."> annotations, and now I have a use case where I have to be able to present those doc strings to the user at run time. Is there a way ATDgen could output the docstrings to OCaml code in a format similar to typeName_OCamlFieldName_DocText, e.g. user_address_doctext?

Copied from original issue: mjambon/atdgen#75

Build and install documentation

The documentation in the subfolder atdgen-doc used to be in its own repository.

Currently the documentation builds if pandoc and make are installed. It is not installed, though. We should install the html and css files into /docs so that they show as https://mjambon.github.io/atd.

atdgen/src/ag_oj_emit.ml", line 1813, characters 17-46: Error: Unbound value Atd_print.string_of_type_expr

#=== ERROR while compiling atdgen.1.12.0 ======================================#
# context      2.0.0~rc | linux/x86_64 | ocaml-system.4.05.0 | file:///home/akochkov/data/tmp/opam-repository
# path         ~/.opam/default/.opam-switch/build/atdgen.1.12.0
# command      ~/.opam/default/bin/jbuilder build -p atdgen -j 7
# exit-code    1
# env-file     ~/.opam/log/atdgen-125727-ca91c7.env
# output-file  ~/.opam/log/atdgen-125727-ca91c7.out
### output ###
# Error: Unbound value Atd_print.string_of_type_expr
# [...]
# File "atdgen/src/ag_ox_emit.ml", line 131, characters 13-30:
# Warning 3: deprecated: String.capitalize
# Use String.capitalize_ascii instead.
#       ocamlc atdgen/bin/.ag_main.eobjs/ag_main.{cmi,cmo,cmt}
# File "atdgen/bin/ag_main.ml", line 395, characters 29-46:
# Warning 3: deprecated: String.capitalize
# Use String.capitalize_ascii instead.
#     ocamlopt atdgen/src/.atdgen.objs/ag_oj_emit.{cmx,o} (exit 2)
# (cd _build/default && /usr/bin/ocamlopt.opt -w -40 -g -I atdgen/src/.atdgen.objs -I /home/akochkov/.opam/default/lib/atd -I /home/akochkov/.opam/default/lib/biniou -I /home/akochkov/.opam/default/lib/easy-format -I /home/akochkov/.opam/default/lib/yojson -no-alias-deps -o atdgen/src/.atdgen.objs/ag_oj_emit.cmx -c -impl atdgen/src/ag_oj_emit.ml)
# File "atdgen/src/ag_oj_emit.ml", line 1813, characters 17-46:
# Error: Unbound value Atd_print.string_of_type_expr

Support for BuckleScript as a new codegen target

I'm using BuckleScript on the client, and ATD on the server (which is fantastic). Unfortunately, BS doesn't have an ATD-like library, which makes converting JSON into OCaml records (and back) very cumbersome.

I'm interested in using ATD to generate the boilerplate for the client in the same way I do for the server.

One caveat is that the browser already has a fantastic JSON-parser, so the generated code is more about plucking the values from the browser-parsed JSON, verifying the types, and returning an OCaml-record.

To give you and example, here's what parsing a JSON-string into a small, four-field record looks like right now (it's using the Reason syntax and BS library, but I hope it's clear enough):

let pluckField decoder field obj =>
  Option.getExn @@ decoder @@ Option.getExn @@ Js.Dict.get obj field;

let decodeJson str handler => {
  open Js.Json;
  let json =
    try (parseExn str) {
    | _ => raise (Failure "Error parsing API response JSON")
    };
  switch (decodeObject json) {
  | Some obj => handler obj
  | _ => raise (Failure "Couldn't decode JSON properly")
  }
};

type health = {accountCount: int, sessionCount: int, sessionsHealthy: bool, accountsHealthy: bool};

let healthOfJsonString jsonString =>
  Js.Json.(
    decodeJson
      jsonString
      (
        fun jsonObj => {
          let accountCount = int_of_float @@ pluckField decodeNumber "accounts_count" jsonObj;
          let sessionCount = int_of_float @@ pluckField decodeNumber "sessions_count" jsonObj;
          let accountsHealthy = Js.to_bool @@ pluckField decodeBoolean "accounts_healthy" jsonObj;
          let sessionsHealthy = Js.to_bool @@ pluckField decodeBoolean "sessions_healthy" jsonObj;
          ({accountCount, sessionCount, sessionsHealthy, accountsHealthy}: health)
        }
      )
  );

I'd like to generate something like the above (well, better eventually) using the ATD machinery. Where would you suggest I start?

hash table or map type

From @dwwoelfel on August 6, 2017 20:13

Is there a way to define a map type where the type of the keys and the values are known, but the key names are unknown?

I have some JSON that looks like

{
  "random-key-1": "string-value-1",
  "random-key-2": "string-value-2"
}

Copied from original issue: mjambon/atdgen#69

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.