neoteroi / essentials-openapi Goto Github PK

View Code? Open in Web Editor NEW

21.0 21.0 8.0 575 KB

Functions to handle OpenAPI Documentation.

License: MIT License

Makefile 0.35% Python 90.17% HTML 9.48%

documentation documentation-generator openapi swagger

essentials-openapi's People

Contributors

Stargazers

Watchers

Forkers

adriangb bitnom kylewm jjedele mh7d anatoliy057 tyzhnenko maia-everett

essentials-openapi's Issues

KeyError when $ref contains invalid characters

When $ref contains invalid characters - including spaces - the code fails with a KeyError.
Improve the code to not raise an exception and handle such references.

Header links do not work when schema name contains dots

This happens in MkDocs, which generates table of contents and links for headings.
When components name contain dots (common case for auto-generated definition files for .NET and Java apps), the link doesn´t work because MkDocs doesn´t maintain the dots.

RequestBody Reference

Marshalling the following spec fails, probably because operation.requestBody.content is null/undefined during template stage. I assume the problem is related to resolving the reference

Spec:

paths:
  "/something":
    post:
      requestBody:
        $ref: "#/components/requestBodies/SomeBody"

components:
  requestBodies:
    SomeBody:
      description: Foo Bar.
      required: true
      content:
        application/json:
          schema:
            type: object
            properties:
              a:
                type: string

Full StackTrace

Traceback (most recent call last):
  File "/opt/homebrew/bin/oad", line 8, in <module>
    sys.exit(main())
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/commands/docs.py", line 51, in generate_documents_command
    generate_document(source, destination, style)
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/generate.py", line 14, in generate_document
    html = handler.write()
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/v3/__init__.py", line 417, in write
    return self._writer.write(
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/jinja.py", line 109, in write
    return template.render(data, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/opt/homebrew/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/v3/views_mkdocs/layout.html", line 14, in top-level template code
    {% include "partial/path-items.html" %}
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/v3/views_mkdocs/partial/path-items.html", line 26, in top-level template code
    {% include "partial/request-body.html" %}
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/v3/views_mkdocs/partial/request-body.html", line 1, in top-level template code
    <p class="request-body-title"><strong>{{texts.request_body}}</strong></p>
  File "/opt/homebrew/lib/python3.10/site-packages/openapidocs/mk/v3/__init__.py", line 260, in simplify_content
    simplified_content = copy.deepcopy(content)
  File "/opt/homebrew/Cellar/[email protected]/3.10.6_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle 'Undefined' object

Cannot generate Markdown file from valid API definition in YAML

I cannot generate Markdown file from properly validated API definition file:

$ oad --verbose gen-docs -s docs/api/swagger.yaml -d api.md
[08/07/23 12:35:24] DEBUG    Running in --verbose mode                                                                                                                                                   main.py:27
                    DEBUG    Reading from file docs/api/swagger.yaml                                                                                                                                   source.py:77
Traceback (most recent call last):
  File "<redacted_path>/.venv/bin/oad", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/openapidocs/commands/docs.py", line 51, in generate_documents_command
    generate_document(source, destination, style)
  File "<redacted_path>/.venv/lib/python3.11/site-packages/openapidocs/mk/generate.py", line 14, in generate_document
    html = handler.write()
           ^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/openapidocs/mk/v3/__init__.py", line 427, in write
    operations=self.get_operations(),
               ^^^^^^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/openapidocs/mk/v3/__init__.py", line 181, in get_operations
    tag = self.get_tag(path_item) or ""
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "<redacted_path>/.venv/lib/python3.11/site-packages/openapidocs/mk/v3/__init__.py", line 219, in get_tag
    tags = operation.get("tags")
           ^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'get'

API definition file swagger.yaml (validated with https://github.com/OpenAPITools/openapi-generator):

openapi: 3.0.3
info:
  title: My API
  description: My API
  version: 1.0.0
servers:
  - url: 'http://localhost:8080/'

paths:
  /client:
    description: Client onboarding operations

    post:
      description: Onboard a new client
      responses:
        201:
          description: Client onboarded successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OnboardedClientDetails'
        400:
          description: Missing attribute in the payload
        403:
          description: Not authorized
        422:
          description: Payload formatted incorrectly
        500:
          description: Internal server error
      requestBody:
        description: Payload to be sent
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/OnboardClientRequest'
      x-amazon-apigateway-integration:
        type: "aws_proxy"
        httpMethod: "POST"
        uri:
          Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:integration-layer-client-management/invocations"
        responses:
          default:
            statusCode: "201"
        passthroughBehavior: "when_no_match"
        contentHandling: "CONVERT_TO_TEXT"
      security:
        - Authorize: [ ]

  /client/{clientId}:
    description: Operations on a specific client

    delete:
      description: Delete a given client
      parameters:
        - in: path
          name: clientId
          required: true
          schema:
            type: string
      responses:
        204:
          description: Client deleted successfully
        404:
          description: Not available or user not authorized to see this client
        500:
          description: Internal server error
      x-amazon-apigateway-integration:
        httpMethod: "POST"
        uri:
          Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:integration-layer-client-management/invocations"
        responses:
          default:
            statusCode: "204"
        passthroughBehavior: "when_no_match"
        contentHandling: "CONVERT_TO_TEXT"
        type: "aws_proxy"
      security:
        - Authorize: [ ]

  /otp:
    description: OTP API key generation

    parameters:
      - in: query
        name: systemId
        required: true
        schema:
          type: string
        description: Client identifier
        example: '3907f492-3af3-4c0d-a09f-81c6ec0e4b3d'
      - in: query
        name: customerId
        required: true
        schema:
          type: string
        description: Customer Id
        example: '121032'

    post:
      description: Generate OTP API key
      responses:
        200:
          description: OTP API key generated successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OTPResponse'
        400:
          description: Missing attribute in the payload
        409:
          description: System already registered with different customerId
        500:
          description: Internal server error
      x-amazon-apigateway-integration:
        type: "aws_proxy"
        httpMethod: "POST"
        uri:
          Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:otp/invocations"
        responses:
          default:
            statusCode: "200"
        passthroughBehavior: "when_no_match"
        contentHandling: "CONVERT_TO_TEXT"
      security:
        - Authorize-otp: [ ]

components:
  schemas:
    OnboardClientRequest:
      type: object
      required:
        - publicKey
      properties:
        publicKey:
          type: string
          description: Public key in PEM format
          example: '-----BEGIN PUBLIC KEY-----\nPUBLIC\nKEY\nLINES\nSEPARATED\nWITH\n-----END PUBLIC KEY-----'
    OnboardedClientDetails:
      type: object
      properties:
        client_name:
          type: string
          description: Name of the client (equal to systemId with added timestamp)
          example: '3907f492-3af3-4c0d-a09f-81c6ec0e4b3d_1685686753'
        clientId:
          type: string
          description: Unique identifier of the client
          example: '33333333334444444445555555555555444555555'
        x5c:
          type: string
          description: x5c (X.509 certificate chain) indicates a chain of one or more PKIX certificates
          example: "<redacted>"
    OTPResponse:
      type: object
      properties:
        otp:
          type: string
          description: OTP API key
          example: '<redacted>'

In my virtual environment (venv) the following PyPi dependencies are used:

mkdocs-material==9.1.21
mkdocs-mermaid2-plugin==1.0.6
python-markdown-comments==1.1.0
essentials-openapi[full]==1.0.8
neoteroi-mkdocs==1.0.4
pymdown-extensions==10.1

My environment:

OS: macOS Ventura 13.5
Python: 3.11.4 (HomeBrew)

Am I missing something?

Snake case to camel case conversion should not happen for Examples!

Fix the bug reported here:
Neoteroi/BlackSheep#173

The examples objects should not be modified to respect camelCase (like other classes where conversion between Python naming convention and OpenAPI Documentation entities that use camelCase).

Can you override template files?

This may be not for this repository and more suited for the neoteroi.mkdocsoad MkDocs plugin but I want to ask anyways just in case.

I was wondering if there was a way to override the template files used by this software for MkDocs. I would like to make some changes to the overall design but I don't know if I could override template files à la theme-extension in MkDocs... And if I can do I lack the knowledge of what might be the proper path for the files to be located in.

I would really apreciate a response here. If that is not doable, would I like to propose it as an idea, so that people could further customize the generated docs using this templating aproach.

Group response codes in tab group on MkDoc output?

Personally speaking, I feel like it would be a bit more useful to have the various Response codes grouped together in a tab group to toggle through.

Having them listed all at once can increase the page length for no real reason, especially given the default OAPI design from swagger having each endpoint in a collapsable section, so this would be the next best option I would say.

Quick mockup to show what I mean:

I understand if this is something not worth considering. Just felt like it would be good to have some option to reduce the vertical height of the generated page, as large APIs could create very large pages...

Empty header when operations don't have any tag.

Use the same example for #27

an empty header is generated "##" because the operations don´t have any tag. Fix to generate a generic text, like "Endpoints".

Show description for schema object fields and responses

Hello! I'm using the OAD mkdocs plugin from https://github.com/Neoteroi/mkdocs-plugins.

It seems that for schema objects, only Name and Type columns are generated. Is it possible to also add a Description column for each object property?

Edit: Descriptions are not displayed for responses either.

Incompatibility in OpenAPI v3 in BlackSheep 2.0.7 and essentials-openapi 1.0.9

Hello,

We have had a BlackSheep app running for over a year. When we attempted to upgrade to 2.0.7, we ran into this error. It seems to be an error in the fundamental OpenAPI class. Since this was working fine until today, I think that it must be a bug in 1.0.9.

Here is the error message:

  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 307, in _get_array_outer_type
    return field_info.outer_type_
AttributeError: 'FieldInfo' object has no attribute 'outer_type_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/application.py", line 726, in _handle_lifespan
    await self.start()
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/application.py", line 715, in start
    await self.after_start.fire()
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/application.py", line 126, in fire
    await handler(self.context, *args, **kwargs)
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/common.py", line 404, in build_docs
    docs = self.generate_documentation(app)
  File "/home/mistral/llm_server_env/datascience-llm-server/app/docs/handler.py", line 34, in generate_documentation
    paths=self.get_paths(app),
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 449, in get_paths
    own_paths = self.get_routes_docs(app.router, path_prefix)
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 1146, in get_routes_docs
    request_body=self.get_request_body(handler),
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 847, in get_request_body
    content=self._get_body_binder_content_type(body_binder, body_examples),
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 821, in _get_body_binder_content_type
    return {
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 823, in <dictcomp>
    schema=self.get_schema_by_type(body_binder.expected_type),
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 642, in get_schema_by_type
    schema = self._get_schema_by_type(child_type, type_args)
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 663, in _get_schema_by_type
    return self._get_schema_for_class(object_type)
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 567, in _get_schema_for_class
    for field in self.get_fields(object_type):
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 739, in get_fields
    return handler.get_type_fields(
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 342, in get_type_fields
    return [
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 345, in <listcomp>
    self._open_api_v2_field_schema_to_type(
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 297, in _open_api_v2_field_schema_to_type
    return self._get_array_outer_type(field_info)
  File "/home/mistral/llm_server_env/lib/python3.10/site-packages/blacksheep/server/openapi/v3.py", line 311, in _get_array_outer_type
    return List[field_info.annotation.__args__[0]]
AttributeError: type object 'list' has no attribute '__args__'. Did you mean: '__add__'?```

WSD file is empty

Breaks with empty component schemas

See:
Neoteroi/mkdocs-plugins#23

Markdown missing a newline after simple response types

When a response either has no content or is a reference to a response schema, the newline after it is getting stripped. Here's a small example:

openapi: 3.0.0
info:
  title: Test
  description: Demonstrate newline bug
  version: 0.0.0
paths:
  /cats:
    get:
      summary: Get a list of cats
      responses:
        "200":
          $ref: "#/components/responses/GetCatsResponse"
    post:
      summary: Create a new cat
      responses:
        "200":
          description: Success
    delete:
      summary: Delete a cat
      responses:
        "200":
          description: Success

components:
  responses:
    GetCatsResponse:
      description: List of cats

Renders markdown like this

❯ oad gen-docs -s openapi.yaml -d output.md --style MARKDOWN

...
### GET /cats
Get a list of cats

### Response 200 OK
Refer to the common response description: [GetCatsResponse](#getcatsresponse)### POST /cats
Create a new cat

### Response 200 OK### DELETE /cats
Delete a cat

### Response 200 OK
...

i.e., ### POST and ### DELETE are not on their own line.

Thanks for taking a look!

Example generation breaks on explicitly enumerated array elements

FastAPI e.g. generates a schema for me which contains a part like this:

"properties": {
          "shape": {
            "title": "Shape",
            "type": "array",
            "items": [
              {
                "type": "integer"
              },
              {
                "type": "integer"
              },
              {
                "type": "integer"
              }
            ]
          },

on which this library then breaks with following error:

  File "/Users/jeff/anaconda3/lib/python3.8/site-packages/openapidocs/mk/v3/examples.py", line 121, in get_example
    return [get_example_from_schema(items) for _ in range(1)]
  File "/Users/jeff/anaconda3/lib/python3.8/site-packages/openapidocs/mk/v3/examples.py", line 121, in <listcomp>
    return [get_example_from_schema(items) for _ in range(1)]
  File "/Users/jeff/anaconda3/lib/python3.8/site-packages/openapidocs/mk/v3/examples.py", line 142, in get_example_from_schema
    schema_type = schema.get("type")
AttributeError: 'list' object has no attribute 'get'

I think I have an easy fix for which I can open a PR.

And in general: Really nice tool! It looks awesome in mkdocs :)

Nullable schema type formatting improvement

It seems there is simply a missing space in the nullable formatting.

I imagine it's supposed to be number(double) | null?

Probably a simple fix to here and here?

"MARKDOWN" style does not create tables using the minimum 3 hyphens

Markdown tables require that we define them using three or more hyphens (---) to create each column’s header. Without three hyphens, certain systems will not interpret this information as a table (for example, DITA-OT transformation to PDF).

Table definition: https://www.markdownguide.org/extended-syntax/#tables

Sample source file: https://github.com/Neoteroi/mkdocs-plugins/blob/main/tests/res/swagger.json

Example command:

oad gen-docs -s json/swagger.json -d docs/swagger.md --style "MARKDOWN"

This is the current Markdown output when using --style "MARKDOWN"

### Cat

| Name | Type |
| -- | -- |                    // This is invalid
| age | integer |
| hunts | boolean |

However, it should in fact be the following:

### Cat

| Name | Type |
| --- | --- |                  // This is valid
| age | integer |
| hunts | boolean |

v3 ValueFormat lacks base64 format

Markdown code spans in description properties gets escaped and therefore not properly rendered in the output

The OpenApiSpecification allows for rich text formatting in description fields using CommonMark Spec. This works well, except when using code spans.

If a code span is used to display some json structure the quotation marks (") around the properties get escaped and therefore not properly rendered in the final markdown file.

For example when executing the following command using the example specification

openapi: "3.0.2"
info:
  title: API Title
  description: |
    **Response**

    ```json
    [
      {
        "a": "b",
        "c": {
          "d": "e"
        }
      }
    ]
    ```
  version: "1.0"
servers:
  - url: https://api.server.test/v1
paths:
  /test:
    get:
      responses:
        '200':
          description: OK

oad gen-docs -s example-openapi.yaml -d test.md --style "MKDOCS"

the following code span in the description of the specification

    [
      {
        "a": "b",
        "c": {
          "d": "e"
        }
      }
    ]

looks like this in the test.md output

[
  {
    &#34;a&#34;: &#34;b&#34;,
    &#34;c&#34;: {
      &#34;d&#34;: &#34;e&#34;
    }
  }
]

My guess is that is related to Jinja autoescape used here https://github.com/Neoteroi/essentials-openapi/blob/main/openapidocs/mk/jinja.py#L77

Maybe it could be resolved by disabling auto escaping for the description sections, using this https://jinja.palletsprojects.com/en/3.1.x/templates/#autoescape-overrides. But not sure if this could lead to other issues down the road.

Add support for specification docs split into multiple files

Add support for specification docs split into multiple files ($ref pointing to files).