interagent / prmd Goto Github PK

Currently you must specify example at array level of the whole array, should be able to infer and just use this top level version as an override.

We should probably see about either writing full on schema validation stuff or pulling in a dependency to do so (instead of the hand-rolled minimal validation of present). Additionally, consider replacing current hand-rolled stuff with validating against a bundled schema. TURTLES!

presently it outputs the schema if it received schema on stdin and nothing if received from a file. Should settle on one or the other.

See https://github.com/digitaloceancloud/api-schema/blob/master/output/doc.md#domain-record-info

overly specific to our use case (and we can just prepend it to overview anyway).

For example, you might want to add extra notes to a resources intro/description or to a particular link. One way might be by virtue of naming convention (ie it could check for app.md to match app.json and maybe even app-list.md or similar).

see also: http://kapeli.com/docsets

For many APIs there may be a better order for resources than alpha-sorted (ie to build from foundational resources up through stuff that builds on top of them). As such it seems like it could be valuable to allow specifying the order explicitly. Perhaps as a comma delimited list or something.

ie perhaps combine wouldn't need meta arg. On the downside, this might make it harder/less useful when trying to run doc against a single resource schema (which seems useful for debugging). It does increase surface area though. It would certainly be easier to add more flexibility here later (than to remove it later). So starting with simple (assume combine) is probably a good call for the near term.

/cc @mmcgrana

All doc examples use api.heroku.com as the host:

https://github.com/heroku/prmd/blob/69eb97bf96eec21ffe1e29aa2ad07fc067db5880/lib/prmd/views/endpoint.erb#L58

@geemus if you have a preferred way you'd like this configured, I'd be happy to submit a PR.

I forget the exact format otherwise. WIth this done, should probably make validator check for (and fail on) defaulted values for these.

$ prmd doc ./schema
foo... ok
bar... ok
baz... failed
# exception displayed below

Right now it errors before stating the file it tried to read

I saw another case (related to chef) where somebody allowed you to provide yaml/json interchangeably. I cringe a little whenever I have to deal with significant whitespace, but it is hard to deny that yaml is easier (and fewer keystrokes) for creating/editing by humans. As a superset of json it also should be pretty straightforward to convert when building the full schema. This might help mitigate some of the desire for a more DSLish approach with a minimal amount of effort involved.

I was working on a schema with a definition like this:

   "address": {
      "description": "address on file for account",
      "type": "object",
      "properties": {
        "addr1": { "type": "string" },
        "addr2": { "type": "string" },
        "city": { "type": "string" },
        "state": { "type": "string" },
        "zip": { "type": "string" },
        "country": { "type": "string" }
      }

And I got an exception here:

https://github.com/heroku/prmd/blob/master/lib/prmd/doc.rb#L93

/Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:93:in `doc_type': undefined method `first' for "object":String (NoMethodError)
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:83:in `block in extract_attributes'
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:44:in `each'
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:44:in `extract_attributes'
        from (erubis:13:in `result'
        from /Users/jroes/.gem/ruby/1.9.3/gems/erubis-2.7.0/lib/erubis/evaluator.rb:65:in `eval'
        from /Users/jroes/.gem/ruby/1.9.3/gems/erubis-2.7.0/lib/erubis/evaluator.rb:65:in `result'
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:146:in `block in doc'
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:121:in `each'
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/lib/prmd/doc.rb:121:in `doc'
        from /Users/jroes/.gem/ruby/1.9.3/bundler/gems/prmd-3d415049e25d/bin/prmd:27:in `<top (required)>'
        from /Users/jroes/.gem/ruby/1.9.3/bin/prmd:23:in `load'
        from /Users/jroes/.gem/ruby/1.9.3/bin/prmd:23:in `<main>'

Which I was able to solve by making it and all of the child type values single-element arrays.

I'm not sure if this is a bug or not, but I was under the impression that the SON schema specs allowed for non-array values here.

This error is happening when running the doc command:

(erubis:28:in `block (2 levels) in result': undefined method `map' for nil:NilClass (NoMethodError)
    from (erubis:25:in `gsub'
    from (erubis:25:in `block in result'
    from (erubis:24:in `each'
    from (erubis:24:in `result'

prmd combine writes a schema.json file, but prmd doc sends markdown to stdout. should they both write to standard out or both write files?

Might be nice to have a way to fill something in for "..." in title when generating a schema with prmd init. I'm not sure if the answer is a project-wide template, command line arg, a combination, or something else.

I don't believe there are reserved words for JSON schema but prmd doc barfs when I have a schema that has a property/definition with the name type.

ie something like app/domain could properly build the nested links

Would provide a centralized place for committee/heroics/etc, as well as consolidating some of the logic already within prmd. Also provides an opportunity to define our own even prettier printing to consolidate things down a bit (ie inlining single element hash/array).

Having -o/--output adds a bunch of code that doesn't seem needed (you can just use bash/whatever to put it into a file if you want, just as easily). ie why require prmd combine schema -o schema.json vs prmd combine schema > schema.json. The later offers more control anyway I suspect and is fewer characters (and less for us to maintain/test).

@cyberdelia @daneharrigan - thoughts?

It would be nice if documentation generation worked against the combined schema, allowing to create documentation if you only have the final combined schema at hand.

or at least as an override/in addition to convention

Current resource plural logic is SUPER naive. Should fix it by either taking plural as argument, allowing plural as explicit extra arg when it is a weird plural or ...?

In particular, single item things should be inline and aligning values would be nice...

Similar to go format, should rewrite an existing schema to have nice key sort order, separating meta data in to a header, etc.

Might be nice to have something like Schema#verify even.

given a spec that looks like this:

{
  "$schema": "http://json-schema.org/draft-04/hyper-schema",
  "id": "schema/domain",
  "title": "... - Domain",
  "type": [
    "object"
  ],
  "definitions": {
    "name": {
      "description": "name of the domain",
      "example": "example.com",
      "format": "string",
      "readOnly": true,
      "type": [
        "string"
      ]
    },
    "id": {
      "description": "unique identifier of domain",
      "example": 32,
      "format": "integer",
      "readOnly": true,
      "type": [
        "integer"
      ]
    },
    "identity": {
      "anyOf": [
        {
          "$ref": "/schema/domain#/definitions/id"
        }
      ]
    }
  },
  "links": [
    {
      "description": "Create a new domain.",
      "href": "/domains",
      "method": "POST",
      "rel": "create",
      "schema": {
        "properties": {
          "name": {
            "$ref": "/schema/domain#/definitions/name"
          }
        },
        "required": [
          "name"
        ]
      },
      "title": "Create"
    },
    {
      "description": "Delete an existing domain.",
      "href": "/domains/{(%2Fschema%2Fdomain%23%2Fdefinitions%2Fidentity)}",
      "method": "DELETE",
      "rel": "destroy",
      "title": "Delete"
    },
    {
      "description": "Info for existing domain.",
      "href": "/domains/{(%2Fschema%2Fdomain%23%2Fdefinitions%2Fidentity)}",
      "method": "GET",
      "rel": "self",
      "title": "Info"
    },
    {
      "description": "List existing domains.",
      "href": "/domains",
      "method": "GET",
      "rel": "instances",
      "title": "List"
    }
  ],
  "properties": {
    "name": {
      "$ref": "/schema/domain#/definitions/name"
    },
    "id": {
      "$ref": "/schema/domain#/definitions/id"
    }
  }
}

I'll get docs like this:

### Domain Delete
Delete an existing domain.

DELETE /domains/{domain_identity}

#### Curl Example

$ curl -n -X DELETE https://api.heroku.com/domains/$DOMAIN_IDENTITY

When I would expect identity to be resolved to id, so it would using "domain_id" instead of "domain_identity". I would also expect if there were two fields used for identity that it would use both, for example domain_id_or_name.

Alternatively I could just be doing something wrong in those weird nested references in link hrefs.

ie avoid a or b or c or d and prefer a, b, c or d

last = description.pop
description = [description.join(", "), last].join(" or ")

see: https://github.com/heroku/prmd/blob/master/lib/prmd/commands/doc.rb#L26

/cc @daneharrigan @mmcgrana

Endpoint templates make assumptions about the usage of an identity definition:

path = link['href'].gsub(%r|(\{\([^\)]+\)\})|) do |ref|
  ref = ref.gsub('%2F', '/').gsub('%23', '#').gsub(%r|[\{\(\)\}]|, '')
  resource = ref.split('#/definitions/').last.split('/definitions/identity').first
  '{' + resource + '_' + schema.dereference(ref)['anyOf'].map {|r| r['$ref'].split('/').last}.join('_or_') + '}'
end

But for non platform API use cases, where there is not on or more possible values here, for example:

"identity": {
      "anyOf": [
        {
          "$ref": "#/definitions/schema/definitions/id"
        }
    ]
}

It would make more sense to be able to declare href as;

"href": "/domains/{(%2Fschema%2Fdomain%23%2Fdefinitions%2Fid)}",

So should we continue making this assumptions or not?

I noticed while trying to validate a JSON schema with prmd today
the following messages:

direwolf $ bundle exec prmd verify docs/api/schema.json
...
docs/api/schema.json: Missing `#/definitions/cloud/readOnly`
docs/api/schema.json: Missing `#/definitions/command/readOnly`
docs/api/schema.json: Missing `#/definitions/exception/readOnly`
docs/api/schema.json: Missing `#/definitions/health/readOnly`
docs/api/schema.json: Missing `#/definitions/job/readOnly`
docs/api/schema.json: Missing `#/definitions/result/readOnly`
docs/api/schema.json: Missing `#/definitions/root/readOnly`
docs/api/schema.json: Missing `#/definitions/run/readOnly`
docs/api/schema.json: Missing `#/definitions/suite/readOnly`

IIUC, this is indicating a missing readOnly property on the resource
description (in this schema we already use readOnly at the attribute
level). Adding e.g. readOnly: false like this made the verification
error messages go away:

--- a/docs/api/schema/cloud.json
+++ b/docs/api/schema/cloud.json
@@ -4,7 +4,7 @@
   "$schema":      "http://json-schema.org/draft-04/hyper-schema",
   "title":        "Heroku Direwolf API - Cloud",
   "type":         ["object"],
-
+  "readOnly":     false,
   "definitions": {
     "api_key": {
       "description":  "a Platform API key encrypted with Fernet",

Is checking for readOnly at the resource level (in addition to the
attribute level) the desired behavior? I don't think I've seen this
before, so wanted to make sure it was intentional before updating
my schemas to conform.

I have a Resource resource and a User resource.

Resource belongs to User, but I do not want to always access Resources through /user/$user_id/resources.
I want to have Resource as an independent resource that can be listed, searched, (...), and I want to be able to access all resources from a user via /user/$user_id/resources.

I can't see a way to handle this correctly with prmd.
What I have done so far is put a link in Resource's schema, of the form:

{
  "description": "Resources of an existing user",
  "href": "/users/{(%2Fschema%2Fuser%23%2Fdefinitions%2Fidentity)}/resources",
  "method": "GET",
  "rel": "instances",
  "title": "Of user"
}

With this link, I have a doc correctly generated in the "Resource" section, with a valid example, but it doesn't feel right.
For instance, if I'd use this schema with Heroics, I'd have to call client.resource.of_user(2).
Using the "Of user" title to have Heroics generate this method for me feels like a trick.

I'd rather call client.user(user_id).resources. So I'd need to have a link in User. I've tried:

{
  "description": "Resources of an existing user",
  "href": "/users/{(%2Fschema%2Fuser%23%2Fdefinitions%2Fidentity)}/resources",
  "method": "GET",
  "rel": "resources",
  "title": "Resources"
}

But then there are several problems:

• prmd doesn't know the "resources" rel, so can't generate proper documentation
• if I use "instances" as the rel value, prmd will assume the link returns instances of Users when it should be Resources
• in Heroics (sorry I always bring it in in prmd issues, or prmd in Heroics issues), that would imply the following call (if I am not mistaken): client.user.resources(user_id)
  This Heroics call is clearly confusing to me, compared to client.user(user_id).resources.

Now that the problem is exposed, I'd like to propose the following:

• I want to be able to use that last link example in my User schema
• prmd should detect when a link's rel attribute is the plural form (inflector FTW ;)) of another resource, and understand it as "instances of [resource]" to be able to fetch correct examples data, attributes, ...
• [Heroics would use the same mechanism to create a "richer" client from the schema]
• the same would apply for the singular form of a resource name in "rel", when used to act on a single nested resource

This would imply that prmd works on the combined schema, but we already make that assumption in at least one place.

As "always", this is a discussion topic and I'd love your thoughts on the subject, in the light of your own use of the tools for Heroku's Platform API (and/or others)...

I am of course ready to work on the implementation of those changes ^^

Add a property to have a machine oriented name for the API rather than relying on title for this.

Require a version property in the schema:

{
    "version": "3.1.1",
}

The usage given by prmd init --help doesn't seem to match how
the command actually works:

$ git show HEAD
commit 74e9520fe3a8edd9fa71fefc3d0332a354c728af
Author: geemus <[email protected]>
Date:   Tue Apr 1 14:55:50 2014 -0500

    make other commands expect combined input

    closes #71


$ bundle exec bin/prmd init --help
prmd init [options] <directory> <resource>
    -m, --meta meta.json             Set defaults for schemata


$ bundle exec bin/prmd init app
# dumps app schema in app.json

So I think the help text should be more like:

$ bundle exec bin/prmd init --help
prmd init [options] <resource>
    -m, --meta meta.json             Set defaults for schemata

Also it should probably explain that it writes the output to stdout.

You can override the contents of a ref locally, so given:

{
  "definitions": {
    "foo": {
      "bar": "baz",
      "qux": "quux"
    }
  },
  "properties": {
    "foobar": {
      "$ref": "#/definitions/foo",
      "bar": "corge"
    }
  }
}

dereferencing #/properties/foobar should result in merging it's context on top of the dereferenced values, which gives us:

{
  "bar": "corge",
  "qux": "quux"
}

pending #72 (to avoid merge conflicts)

Ideally we should be able to render one or more templates in the examples section. At first this can just replace the existing curl stuff, but eventually it would be nice to be able to generate with, say, a ruby template. And/or getting to a point where you could click on things to switch between different language examples.

ie explain what identity is, maybe with an example

Currently this is built from all the individual example values for attributes, being able to explicitly override this seems like it (might) be useful. Should consider when/why.

For example with nullable attributes (that are commonly null) having the example with the description be null might be good, whereas you might not want to either omit that key/value pair altogether in the curl example (or provide a non-null value).

Seems like a good idea. Not sure how best to do this/fit it in though.

Consider adding something like #/definitions/shared/created_at, #/definitions/shared/id, #/definitions/shared/updated_at, etc. (shared could also be 'common' or something). These can then be referenced in sub-schemas (and then you can just override stuff from common/shared as needed). Not really required, but certainly might be nice-to-have.

/cc @olance

even just [] going to data[] would be awesome