Giter Club home page Giter Club logo

semoasa's Introduction

Semoasa - Specification Extension Metadata for OAS Annotations (DRAFT)

Background

The OpenAPI Specification, formerly known as the Swagger Specification, is a standard, machine-readable format for REST-style API definitions and documentation. Swagger 2.0 introduced vendor extensions, which allow certain objects to have properties named with an x- prefix, with arbitrary or independently specified property values. Vendor extensions were later renamed to specification extensions in OpenAPI 2.0 and 3.0.

Motivation

While specification extensions are widely used, there is no standardized way to define their syntax, expected values, and allowed usage context. This makes it much more difficult for OpenAPI tools to support specification extensions with content assist, validation, and other features that those tools typically provide for standard OpenAPI language constructs.

For example, an OpenAPI editor will usually have design-time knowledge of the OpenAPI specification, and may make use of a JSON Schema that encodes some of this knowledge. Based on this knowledge, it can provide context-sensitive editing assistance and validation. But if the user adds extended properties with the x- prefix, the trail goes cold. Unless the editor has been designed with explicit prior knowledge of that specification extension, it can't tell the user what the extended property means, what kinds of values are expected, and whether that extension is even meaningful in this context.

Semoasa is a machine-readable format for specification extensions that provides a base level of structural and descriptive information, along with optional links to external resources for human-readable documentation. Like OpenAPI itself, specification extensions may have certain constraints and semantics that aren't captured in a schema or metadata descriptor; but the information carried in Semoasa is sufficient to allow OpenAPI tools to provide a user experience much closer to what they provide for standard OpenAPI syntax.

Use Cases

Roles

Semoasa use cases involve interactions among systems or system implementors in different roles:

  • The extension provider (or simply provider) defines one or more specification extension properties, and usually provides software that uses these properties. The provider may be a vendor, an open source software project, or a standards body (formal or informal). In this scenario, the provider may specify a URL as a canonical source location to access the Semoasa document.
  • The publisher exposes a Semoasa document describing a single specification extension property or a catalog of extension properties. The publisher may be an extension provider, publishing its own extension properties in a Semoasa document. Or it may be a registry, publishing a catalog of third-party extensions.
  • The consumer is a software program that reads a Semoasa document, and makes use of the specification metadata. Consumers may be OpenAPI editors, validators, code generators, automated build tools, middleware integration modules, or other processors that work with OpenAPI documents.

Catalog Patterns

Publishers and consumers may want to exchange catalogs in different forms:

  • Directory Catalog - A publisher may provide a list of known OpenAPI specification extensions, referring back to the canonical source locations for the actual extension metadata. A consumer of a Directory Catalog would need traverse references to those locations to retrieve the metadata.
  • Aggregation Catalog - A publisher may provide a collection of known OpenAPI specification extensions, including the full extension metadata. This allows a consumer to ingest a full set of metadata without requiring additional round trips. In this case, it should still be possible to include the canonical location, so the consumer can refresh these definitions as needed.

Tool Functions

Semoasa is designed with a few representative use cases in mind

  • Content Assist - Providing in-place code assist, or "Intellisense", including proposals for specification extension properties intended for use in the current context.
  • Tool Tips - Extension property descriptions and/or formatted documentation, visible when hovering over an actual or proposed usage of the extension property.
  • Validation - Checking usage of extension properties against the provided extension descriptor, flagging errors or warnings where an extended property appears in an unexpected context, or has a value that does not conform to the specified schema.

Design Goals

Semoasa aims to meet the following goals, to the extent practical:

  • Consistency with OpenAPI, so developers familiar with current OpenAPI concepts, terminology and style will be familiar with corresponding elements of Semoasa.
  • Common Technology Stack with OpenAPI, so that essential functions like YAML editing, schema validation, and JSON reference resolution can be performed by the same components used with OpenAPI tool implementations.
  • Support for OpenAPI 2.0 and later, to ensure relevancy with current and future usage.
  • Provider-Neutrality, being driven by the community, and not being unduly influenced by a particular set of tools or OpenAPI specification extension providers.
  • Extension Identity, allowing consumers to disambiguate duplicate names. Consumers may import extension metadata from multiple publishers, with overlapping catalogs. Since extension providers are not required to reserve unique property names in a central registry, name clashes can occur. To meet the identity requirement, Semoasa introduces namespace to define a set of extension properties under coordinated management.

Structure

Once the Semoasa format gets some traction and achieves a level of stability, we'll write a formal specification. For now, this outline format summarizes the structure of a Semoasa document:

  • Extensions Object - an object defining one or more extensions, organized by namespace.
    • openapiExtensionFormat - a version string indicating the format used to describe the extensions. Allows tools to support multiple versions of specification extensions. Each revision of the specification will prescribe a specific version to use. Should adhere to semantic versioning, allowing tools to support a given major or minor version.
    • [namespace] (patterned field, type: Namespace Object or reference) - a namespace encompassing a set of managed Specification Extensions. Within a given namespace, management of Specification Extensions is expected to be quasi-coordinated, at least to the extent of ensuring unique extension names. As a convention, namespaces should generally use reverse-DNS notation, with additional segments as needed, e.g. com.adobe.experience.public. (Pattern for this set of properties can enforce that convention, and thus exclude statically defined properties like openapiExtensionFormat.) May contain a set of Extension Descriptors, or a JSON Reference to a Namespace Object, to support the Directory Catalog use case.
      • [extensionPropertyName] (patterned field, type: Extension Object or reference)- name of the specification extension property. must start with "x-" as per the OAS specification. We may want to support wildcards or template parameters in the name. May contain an Extension Object or a JSON Reference thereto, supporting the Directory Catalog use case (one level deeper than namespace).
        • summary - a plain-text summary of the purpose of the extension.
        • description - markdown documentation describing the extension.
        • deprecated - a Boolean to indicate whether this extension is deprecated and should no longer be used. If the extension property is deprecated, the summary or description SHOULD specify the reason, and/or point to another document defining an extension property that replaces this one.
        • externalDocs (type: External Document Object or reference) - points to an external human-readable specification. May be specified inline, or may be a JSON reference to an ExternalDocumentation object defined in the same document (usually in a designated components location), or an external document.
          • description - markdown description of the external documentation resource.
          • url - URL resolving to the external documentation resource.
        • location - URL that resolves to the authoritative definition of the OAS Extension descriptor. Supports the Aggregation Catalog use case, providing the full specification along with a canonical location.
        • provider (type: Provider Object or reference) - specifies some meta-information about the party responsible for defining the Extension. May be specified inline, or may be a JSON reference to a reusable component.
          • name - name
          • (other) - other properties, TBD
        • schema (type: Schema Object or reference) - an OAS 3.0-compliant Schema Object describing the expected value of the extension property. May be specified inline, or may be a JSON reference thereto.
        • oas2 (type: OAS2 Context Object) - Specifies usage in OAS2 context.
          • usage - specifies allowed usage as one of the following:
            • prohibited - may not be used in oas2.
            • unrestricted - may be used in any oas2 object that allows specification extension properties.
            • restricted - may be used only in objects of the types specified in the 'objectTypes' property.
          • objectTypes (value: array of OAS2ObjectTypeName enum) - A list of OAS2 object types in which the extension property may be used. Valid only if 'usage' is RESTRICTED.
        • oas3 (type: OAS3 Context Object) - Specifies usage in OAS3 context.
          • usage - specifies allowed usage as one of the following:
            • prohibited - may not be used in oas3.
            • unrestricted - may be used in any oas3 object that allows specification extension properties.
            • restricted - may be used only in objects of the types specified in the 'objectTypes' property.
          • objectTypes (value: array of OAS3ObjectTypeName enum) - A list of OAS3 object types in which the extension property may be used. Valid only if 'usage' is RESTRICTED.
    • components (type: Components Object) - collections of reusable components.
      • schemas (type: Schemas Object) - catalog of reusable Schema Objects.
        • [schemaName] (patterned field, type: Schema Object) - addressable object name.
      • providers (type: Providers Object) - catalog of reusable Provider Objects.
        • [providerName] (patterned field, type: Schema Object) - addressable object name.
      • externalDocs (type: External Docs Object) - catalog of reusable External Documentation Objects.
        • [documentName] (patterned field, type: Schema Object) - addressable object name.

Examples

Here's a starting example:

openapiExtensionFormat: 0.1.0
com.amazon.aws:
  x-amazon-apigateway-integration:
    summary: Specifies the integration of the method with the backend.
    description: |
      Specifies details of the backend integration used for this method. 
      This extension is an extended property of the Swagger Operation object.
      The result is an API Gateway integration object.
    externalDocs:
      description: AWS documentation page in the  Amazon API Gateway Developer Guide 
      url: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html
    provider:
      name: Amazon Web Services
      url: https://aws.amazon.com/
    schema: 
      type: object
      properties:
        cacheKeyParameters:
          type: array
          items:
            type: string
          description: A list of request parameters whose values are to be cached.
        cacheNamespace:
          type: string
          description: An API-specific tag group of related cached parameters.
        credentials:
          type: string  
          description: |
            For AWS IAM role-based credentials, specify the ARN of an appropriate IAM role. 
            If unspecified, credentials will default to resource-based permissions that must
            be added manually to allow the API to access the resource. For more information,
            see Granting Permissions Using a Resource Policy. 
        contentHandling: 
          type: string
          description: |
            Request payload encoding conversion types. Valid values are 
            1) CONVERT_TO_TEXT, for converting a binary payload into a Base64-encoded string 
            or converting a text payload into a utf-8-encoded string or passing through the 
            text payload natively without modification, and 
            2) CONVERT_TO_BINARY, for converting a text payload into Base64-decoded blob or 
            passing through a binary payload natively without modification.
        httpMethod:
          type:  string
          description: |
            The HTTP method used in the integration request. For Lambda function invocations, 
            the value must be POST.
        #... (other properties)
    oas2:
      usage: restricted
      objectTypes:
      - OperationObject
    oas3:
      usage: restricted
      objectTypes:
      - OperationObject

Implementations

Further implementations coming soon, pending review and adoption of the specification by tool providers.

Project Description Language Website/Example
widdershins OpenApi / Swagger, AsyncApi & Semoasa definition to Slate / Shins compatible markdown Node.js APIs.guru extension documentation

Resources

Contributing to Semoasa

Please comment! This is an early DRAFT specification. We need input from OAS tool providers, extension providers, the OpenAPI TDC, and other interested parties.

Please review the open issues, feel free to comment on these, and open new issues.

semoasa's People

Contributors

mikeralphson avatar tedepstein avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

mikeralphson

semoasa's Issues

More visibility for project / cross-spec development work

Hi @tedepstein - some discussions have taken place within the OpenAPI and AsyncAPI communities where we'd very much like something like Semoasa to have a place in defining a machine-readable format for API specification extensions.

In time we may be able to find a home within one of those communities for the Semoasa specification, but in the near term, would you be interested in possibly transferring the repo to a cross-spec working group where we can continue development? There is also a Slack instance for cross-spec activity.

Round-tripping to RAML 1.0 annotations

From a brief reading of the relevant part of the spec all we may be missing is an optional displayName.

A friendly name used only for display or documentation purposes. The default is the element key, the name of the annotation itself.

As this seems distinct from the summary, and description maps 1:1.

A raml1 - RAML1 Context Object may not be necessary if the allowedTargets can be deduced from whichever oas2 or oas3 one is converting from.

Consider aligning context objectType enums with OAS spec. fragment-ids

I.e. in the example, OperationObject -> operationObject. The rationale for this is that fragment identifiers are case-sensitive. This would aid linking to the relevant area of the specifications.

It might also help those who want to produce (and keep up to date) a combined specification from the source OAS specification plus their extensions. Cf SmartAPI (ping @newgene).

An alternative would be to use the property names from the OpenAPI schemas (once an official 3.0.0 schema is settled upon), thus OperationObject -> operation for OASv2 or the gnostic 3.0 schema, -> Operation for webron's 3.0 schema. This wouldn't be my favoured option, as the schema may impose its own model (e.g. ParameterWithSchemaWithExample).

Consider three-level identity: namespace.extension.property

The current spec uses a namespace and a property name to establish unique identity. So the fully qualified name of an extension property would be namespace.property. In the readme example, the FQN of the property would be com.amazon.aws.x-amazon-apigateway-integration.

Should we introduce an extension object as an extension property or closely related set of extension properties defined within a namespace?

That would give us something like this:

openapiExtensionFormat: 0.1.0
# namespace
com.amazon.aws:
  # extension
  apiGateway:
    name:  Amazon API Gateway Extension to OpenAPI
    summary: The API Gateway extension supports the AWS-specific authorization and API Gateway-specific API integrations.
    externalDocs:
      description: AWS documentation page in the  Amazon API Gateway Developer Guide 
      url: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions.html
    # extension property
    x-amazon-apigateway-integration:
      summary: Specifies the integration of the method with the backend.
      description: |
        Specifies details of the backend integration used for this method. 
        This extension is an extended property of the Swagger Operation object.
        The result is an API Gateway integration object.
      externalDocs:
        description: AWS documentation page in the  Amazon API Gateway Developer Guide 
        url: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html
    provider:
      name: Amazon Web Services
      url: https://aws.amazon.com/

In this structure, the FQN would be com.amazon.aws.apiGateway.x-amazon-apigateway-integration.

Pros

OpenAPI uses the term "specification extensions," to communicate extensibility through x- prefixed properties. But it doesn't define how we should use that term to refer to a concrete thing. I would think of a specification extension as a written spec (machine- and/or human-readable) that defines one or more extension properties. It might be nice to formalize that meaning here, so we can talk about "specification extensions" as real things that have an unambiguous meaning, and a well-bounded representation in our format.

(Along the way, I've found myself referring to extension properties, though I haven't formally defined that term. I think we need that bit of terminology. We wouldn't want to say "specification extension" every time we refer to one of these properties, because it's too abstract, and because the term "specification extension" just doesn't sound like it means a single property. That term wants to refer to a spec that defines one or more such properties.)

As shown in the example above, the Extension Object can have its own name, summary, externalDocs, and possibly other metadata.

Cons

Even without defining Extension as its own thing, it's possible for a provider to use hierarchical namespaces. So Amazon could define extension properties under the namespaces com.amazon.aws.apiGateway, com.amazon.aws.lambda, etc. From this perspective, the extension starts look like it's just the last segment of the namespace.

Introducing extension as another level in the taxonomy could be seem like too much structural weight, or overhead, for a provider who wants to define a single a single extension property.

We could solve the terminology problem by:

  • making specification extension synonymous with extension property; or
  • defining specification extension as the set of extension properties defined within a namespace; or
  • ignoring the terminology problem and muddling through.

Break out the structure into a formal spec

Right now, the structure section in the README serves as a placeholder for a specification. Once this starts to settle, I would want to write that formal spec.

We might keep a Structure section in the README as a quick reference guide, if it's not too much maintenance overhead. (Extra brownie points for auto-extracting the summary from the spec markdown.)

Consider using collection properties for namespaces & extension properties

I have so far resisted using namespaces to group the Namespace Objects. Since namespace properties use reverse-DNS notation, the SEMOASA schema (when we get around to creating it) can use patternProperties to distinguish them from statically defined properties.

Similarly, there's no extensionProperties collection to hold the Extension Property Objects within a namespace. Right now, these Extension Property Objects are the only properties of a namespace. And if needed, we could again use patternProperties to distinguish these x- prefixed property names from other statically defined properties.

If we introduced collection properties, we'd have something like this:

openapiExtensionFormat: 0.1.0
namespaces:
  com.amazon.aws:
    extensionProperties:
      x-amazon-apigateway-integration:
        summary: Specifies the integration of the method with the backend.
        description: |
          Specifies details of the backend integration used for this method. 
          This extension is an extended property of the Swagger Operation object.
          The result is an API Gateway integration object.
        externalDocs:
          description: AWS documentation page in the  Amazon API Gateway Developer Guide 
          url: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html
     ...

Should we use collection properties?

Pros

  • Consistency with OpenAPI style.
  • Internal consistency, since there are cases where we need collection properties:
    • objectTypes
    • schemas
    • providers
    • externalDocs
    • ... and maybe others, as the spec evolves.
  • It makes the format less fragile, because we could conceivably introduce sibling properties that would break our patternProperties assumptions, or force us to define an even more specific regex to distinguish a certain group of properties.
  • It might make things easier for parsing and in-memory object representation of a SEMOASA document in strongly-typed languages like Java, C#, etc. If the collection properties make a significant difference in this area, that almost certainly tips the scale in favor of using them.

Cons

I don't like collection properties. If find them, and the extra indent levels they introduce, kind of cumbersome and verbose. They're one of my gripes about YAML, or the use of YAML as a simplified DSL for JSON.

And if we choose to add extension as another level in the hierarchy, as suggested in #1, it exacerbates this exploding indent effect.

All that said, I will bow to reality, if I have to.

Suggest adding a (cut-down) info object at the root

Which might look something like:

Fixed Fields
Field Name Type Description
title string REQUIRED. The title of the SEMOASA documentation.
description string A short introduction to the documentation. CommonMark syntax MAY be used for rich text representation.
contact Contact Object The contact information for the documented extensions.
version string REQUIRED. The version (or date in ISO-8601 format) of the SEMOASA document (which is distinct from the openapiExtensionFormat property.

Confirm use of OpenAPI's Schema Object

The spec says that extension property schemas should be OpenAPI Schema Objects. At first blush, this seems consistent with this stated design goal:

Common Technology Stack with OpenAPI, so that essential functions like YAML editing, schema validation, and JSON reference resolution can be performed by the same components used with OpenAPI tool implementations.

But, while Schema Object is implemented by certain OpenAPI tools and components (parsers, documentation UIs, code generators, etc.), it's not part of the technology stack used to implement those tools and components.

Pros

  • Puts limits on the use of certain features of JSON schema. (Both a 'pro' and a 'con')
  • Adds its own syntax and semantics for discriminators, format, etc. (Both a 'pro' and a 'con')
  • Lets us use the OpenAPI convention of allowing markdown in description in top-level schemas, property sub-schemas, etc. This could be useful if we want to use SEMOASA to produce rich-formatted documentation of Specification Extensions, though that is not stated as one of the intended use cases.

Cons

  • Puts limits on the use of certain features of JSON schema. (Both a 'pro' and a 'con')
  • Adds its own syntax and semantics for discriminators, format, etc. (Both a 'pro' and a 'con')
  • Libraries that enforce these constraints, implement these extended semantics, and render markdown descriptions may not be readily available, except in OpenAPI tools. Making them available in a lower-level stack, used by tool providers to create those tools might require significant refactoring. It might just not be practical to do this.

Variable schemas for OASv2 and OASv3 (edge case)

My example is Microsoft's x-ms-paths (docs). Where x-ms-paths is an analogue of the paths property (i.e. it contains pathItem Objects).

I'm assuming MS will continue to use x-ms-paths with OASv3, as the problem it 'solves' is unchanged in OASv3. So in OASv2 it will be structured like a container of OASv2 pathItem Objects and OASv3 pathItem Objects in OASv3.

So, would something like the following be valid?

#...
com.microsoft:
  x-ms-paths:
  #...
    schema:
      oneOf:
      - $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/schemas/v2.0/schema.json#/definitions/Paths
      - $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/49e784d7b7800da8732103aa3ac56bc7ccde5cfb/schemas/v3.0/schema.yaml#/definitions/Paths

And does it provide enough information as to the context for the use of each schema? Without some kind of hint, it seems to me that the schema would be only really suitable for validation, not auto-completion (generated documentation would be partially useful).

I think an OASv2 schema object is a proper subset of the OASv3 schema object, unless it references type: file or discriminator.

One option would be to have a schema property under the oas2 and oas3 properties which overrides the default schema. Then each could be defined as their appropriate type from the relevant spec. version (refs #4, #8 (comment)), and we gain a mechanism to support OASv4 etc. It would still be up for debate as to whether the top-level schema should be JSON-Schema draft X or some version of OpenAPI-Schema.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.