ietf-tools / author-tools Goto Github PK

Install Roboto mono fonts required by xml2rfc on docker instance.

Go to https://fonts.google.com/specimen/Roboto+Mono, and download the font. Follow the installation instructions above, as applied to this download.

API docker instance must install required fonts for xml2rfc.

"Error converting first draft to text: Expected to have the current year when month is missing, but found '2021'"
This is a problem. I need to compare a version from last year with a new version, so of course last year's version has last year's date. For purposes of comparison, the date rule should not be enforced at all. I might need to compare a 2020 version with a 2021 version in 2022!

Add switches to control idnits output.
Default switches should follow current idnits web service.

Example: https://www6.ietf.org/tools/idnits?url=https://www.ietf.org/archive/id/draft-segers-tls-cert-validation-ext-00.txt

Reference: https://github.com/ietf-tools/idnits-pyht/blob/main/idnits.pyht#L101

The API should have an option to turn of "ignore whitespace" flags.
Depends on ietf-tools/iddiff#9

In api.yml, under securitySchemes -> apiKeyAuth perhaps a description such as:

description: API key to authorize requests. Can be found at https://datatracker.ietf.org/accounts/apikey

Right now we only appear to support extracting ANBF from text documents.

From @alicerusso:

For extracting, would be nice if it allowed XML as input, and then extracted the sourcecode type="abnf" (i.e., rfclint --extract="abnf" -n).

Change URL parameter names in iddiff from id_? to doc_? because these supports both IDs and RFCs.

Integrate with datatracker /api/appauth/authortools API endpoint to authenticate API users.
Datatracker ticket: https://trac.ietf.org/trac/ietfdb/ticket/3396

Add x86_64-linux to Gemfile.lock PLATFORMS.

bundle lock --add-platform x86_64-linux

Ref: #84 (comment)

As per RFC6648, use of 'X-' prefix is deprecated.
But X-API-KEY seems to be the commonly used header name for API KEYs.
Change the X-APIKEY to X-APIKEY.

This might come with some risks, but it might be good to allow users to specify a URL for a document. This is a feature that I routinely use with rfcdiff, using input URLs on GitHub pages or in GitHub repositories with raw.githubusercontent.com.

When an unexpected error occurs, API should provide more information about the tool that caused the unexpected error.

See ietf-tools/ietf-at-ui#87

idnits script relies on wget but wget is missing in the current docker instance.
Install wget.

rjsparks@unformal tmp % curl -X 'GET' \
  'https://author-tools.ietf.org/api/version' \
  -H 'accept: application/json'
{"error":"API key is missing"}

I suspect this endpoint should be open.

The docs currently say:

apikeystring | API Key (can be submitted as query string)

Is that really true?

Description

Author Tools convert xml2rfc v2 documents to v3 under the hood.
Add a warning when this happens.

Code of Conduct

• I agree to follow the IETF's Code of Conduct

With iddiff v0.2.0, by default new lines with only whitespace characters don't get shrink anymore.
Add an API option to use the -s switch to shrink the whitespace characters.

In xml2rfc.tools.ietf.org a reload of the page reloads the sorce from disc and recompiles the document. This is very practical and quick. Would be very good if this could be added to authors-tools.

I think not all the tools that kramdown-rfc2629 supports are installed. You can use https://github.com/larseggert/i-d-toolchain/blob/main/draft-i-d-toolchain-test-00.md to see what you need to install, that document executes all the currently supported converters.

(You can also look at https://github.com/larseggert/i-d-toolchain/blob/main/Dockerfile for how to install them.)

Error output of tools like xml2rfx includes a line number.
At the moment Author Tools doesn't show the line number.

Example:
Author Tools:

xml2rfc error: Inconsistent table width: Found different row lengths in this table

xml2rfc:

draft-example-original.xml(428): Error: Inconsistent table width: Found different row lengths in this table

428 is the line number.

API authentication related error responses are undocumented in the OpenAPI spec.
Reference: #20 (comment)

Diff with latest is very handy for an author working solo. However, it's quite a nuisance when working with co-authors, because it doesn't provide a self-contained HTML file like rfcdiff does. I often want to generate the diffs as a single file I can send to my co-authors.

Could we have a mode for Diff with latest that generates a stand-alone file? (Exactly as iddiff does.)

Without this I have no choice but to generate txt files myself and run rfcdiff or iddiff separately.

When I convert a rather old XML file to XML, it does what I expect: converts v2 to v3. However, it doesn't tell me that's what it's doing. If we want to encourage people to switch to v3, it should probably announce that it has done the conversion and say "please use the new format from now on".

/api/idnits have verbose turned on by default.
This should be off by default.

Review and align default options for xml2rfc operations based on default options on xml2rfc.

Reference: https://trac.ietf.org/trac/xml2rfc/browser/trunk/cli/xml2rfc/run.py

Documents with local file system entities generate 500 errors.
Instead, the author tools API should return the appropriate error.

When just document name without version is provided for comparison, author tools doesn't fetch the previous document.

Example:

• https://author-tools.ietf.org/diff?doc_1=draft-carpenter-rfced-iab-charter
• https://author-tools.ietf.org/diff?doc_1=draft-ietf-stir-certificates

It is common to find errors in the source when using the tool, which requires one to go fix them and update the source file. But after the file is updated and you click on any of the buttons, it throws the error - TypeError: Failed to fetch!

Even choosing the file again or a refresh doesn't fix it!

I need to open the https://author-tools.ietf.org/ in a new tab to make it work!

Browser: Chrome

Set environment variable XML_RESOURCE_ORG_HOST=bib.ietf.org to use bib.ietf.org as default XML resource host for kramdown-rfc.

idnits verbose output shows document path information, this should be scraped out.

Author Tools should provide idnits results similar to https://www6.ietf.org/tools/idnits?url=https://www.ietf.org/archive/id/draft-segers-tls-cert-validation-ext-00.txt

Description

It would be nice to have a page that documents the settings and defaults for the xml2rfc interface (e.g., is --id-is-work-in-progress set? Is --add-xinclude set? etc.).

On the https://author-tools.ietf.org/about.html page, maybe a column called "Settings" could be added that has links to pages that explain the configurations (scope creep: documenting the configurations of the other tools listed on that page).

Code of Conduct

• I agree to follow the IETF's Code of Conduct

Expand /api/version API endpoint by adding version details of additional dependencies / tools like Weasyprint, GoAT etc.

Hi,

The following error is displayed -

{"error":"github.com domain is not allowed."}

when I try to input GitHub URL for comparison! Such as https://author-tools.ietf.org/diff?doc_1=draft-ietf-opsawg-sap-00&url_2=https://github.com/IETF-OPSAWG-WG/lxnm/raw/master/I-D-sap/draft-ietf-opsawg-sap.txt

Since a lot of development happens on GitHub in IETF (and IETF encourages it), this check on the domain seems to be a mistake

The language on the main page, like "Plain text as .txt", might be read to imply that filename extensions are critical. Would a ".text" file be ignored even if it was submitted as "text/plain"?

It might be enough to say "Plain text (.txt)" or "Markdown (.md, .mkd, or .markdown), in either kramdown-rfc2629 or mmark format".

Ideally, the tool would detect the file type according to some clear, documented rules. I would suggest that Content-Type take precedence over filename, with the filename only being used when the content-type rule is inconclusive. That's two small lookup tables. I would not do content-type sniffing, as that seems unlikely to be necessary.

(The only reason I say that file extension is needed is that Windows still doesn't associate the correct media type with .md files.)

Description

Implement Prometheus metrics using https://github.com/prometheus/client_python

Code of Conduct

• I agree to follow the IETF's Code of Conduct

From @alicerusso about the ABNF validation service

Please display "No errors during parsing." or similar when that is the case.

The stopgap page (thank you for that) and the old page displayed "No errors during parsing." at the top.

Author-tools output when parsing is successful is display of the input, listing each UNDEFINED.

It's not clear that BAP found no errors in this case - a clear indication that everything is OK should be added.

Author Tools uses the filename to determine the document name when only a single document is given for iddiff comparison.
Whenever an XML file is available Author Tools should use the docname attribute under rfc element to determine the document name.

This document name is fed into datatracker API to figure out the latest draft in that series.

Description

Add the ability to post a file or choose a document name on /api/idnits endpoint

Code of Conduct

• I agree to follow the IETF's Code of Conduct

I really preferred the tools version that would show the output in the browser without any other clicks. If you are debugging, your cycle is fix the text, submit, look at the result, repeat. That should be two clicks (submit and back). It could be one click with proper windows.

The new iddiff tool shows spurious diffs - apparently in cases where page breaks occur in different places in the two documents.

Here is an example using the new tool:
https://author-tools.ietf.org/diff?doc_1=rfc8920&url_2=https://www.ietf.org/archive/id/draft-ppsenak-lsr-rfc8920bis-00.txt

Here is an example using the old rfcdiff tool on the same two files:
https://tools.ietf.org//rfcdiff?url1=rfc8920&url2=https://www.ietf.org/archive/id/draft-ppsenak-lsr-rfc8920bis-00.txt

Accept API key in headers as alternative for form data.

Both the RFC-Editor and the datatracker webpages use unpaginated as the plain text format nowadays. Author-tools should just do the same. As unpaginated is the new IETF standard format for plain text, it is a bit strange that author-tools does not even support it.

(But please keep paginated text as an option. I would like a text+pdf option as well)

Describe the issue

Author Tools doesn't capture xml2rfc v3 reference errors where .xml extension is missing.

Example reference:

<reference anchor="RFC9110" target="https://www.rfc-editor.org/info/rfc9110">

xml2rfc output:

$ xml2rfc drafts/draft-ietf-httpbis-http2bis.xml
Error: Failure fetching URL https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.9000 (HTTPSConnectionPool(host='xml2rfc.tools.ietf.org', port=443): Max retries exceeded with url: /public/rfc/bibxml/reference.RFC.9000 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))))
Error: Failure fetching URL https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.9000 (HTTPSConnectionPool(host='xml2rfc.tools.ietf.org', port=443): Max retries exceeded with url: /public/rfc/bibxml/reference.RFC.9000 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))))
Warning: The v3 formatters require full explicit URLs of external resources.  Did you forget to add '.xml' (or some other extension)?
Error: Unable to resolve external request: "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9000"
Error: XInclude processing failed: could not load https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.9000, and no fallback was found, line 3032
Unable to complete processing drafts/draft-ietf-httpbis-http2bis.xml

Author Tools doesn't capture the above error.

Code of Conduct

• I agree to follow the IETF's Code of Conduct

Add support for mmark.
Mmark markdown file can be identified by the first line read, that starts with:

%%%

In rfcdiff, users can construct a URL like https://www.ietf.org/rfcdiff?url2=draft-ietf-stir-certificates-18
This results in the provided draft being on the right-hand side.

Author Tools iddiff alternative is using id_1 or url_1 (for URL).
The provided document will end up on the right-hand side.

Author Tools iddiff should support specifying just id_2 or url_2.
And the provided document should be on the right-hand side on the diff results.

See #66 for parameter name changes.

Implement ID validation API endpoint at /api/validate
This must accept the same input formats that /api/render/ accepts and output will be a JSON output with warnings, errors and idnits output

Disable requirement for datatracker API token to access author tools API

From @alicerusso about the ABNF validation service:

For extracting from text, could the search for instances of "=" be changed to " = " (with space on each side) and " =/ "? sample input: RFC 9253. seems to extract every line that contains "=" (yielding lots of non-ABNF lines).

We should sample IDs to see if they are careful with spaces around = and =/. If there are any that don't have the spaces, then perhaps we should look at making looking for such spaces a runtime-option.